-- Logs begin at Mon 2022-07-04 05:06:40 UTC, end at Mon 2022-07-04 06:40:23 UTC. -- Jul 04 05:39:58.171113 dv-cinder-797333-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 04 05:39:58.180081 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 4 05:39:58 2022] *** Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: nodename: dv-cinder-797333-5-iscsi Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: machine: x86_64 Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: clock source: unix Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: pcre jit disabled Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: detected number of CPU cores: 4 Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: current working directory: / Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: detected binary path: /usr/bin/uwsgi-core Jul 04 05:39:58.182098 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: your processes number limit is 47680 Jul 04 05:39:58.182672 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: your memory page size is 4096 bytes Jul 04 05:39:58.182672 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: detected max file descriptor number: 2048 Jul 04 05:39:58.182672 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: lock engine: pthread robust mutexes Jul 04 05:39:58.182672 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: thunder lock: enabled Jul 04 05:39:58.182672 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 04 05:39:58.182672 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 04 05:39:58.225877 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: Python main interpreter initialized at 0x55628d84e9c0 Jul 04 05:39:58.225877 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: python threads support enabled Jul 04 05:39:58.225877 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: your server socket listen backlog is limited to 100 connections Jul 04 05:39:58.225877 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: your mercy for graceful operations on workers is 90 seconds Jul 04 05:39:58.226234 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: mapped 403077 bytes (393 KB) for 2 cores Jul 04 05:39:58.226301 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: *** Operational MODE: preforking *** Jul 04 05:39:58.226557 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: *** uWSGI is running in multiple interpreter mode *** Jul 04 05:39:58.226557 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: spawned uWSGI master process (pid: 80118) Jul 04 05:39:58.226765 dv-cinder-797333-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 04 05:39:58.231350 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: spawned uWSGI worker 1 (pid: 80128, cores: 1) Jul 04 05:39:58.231350 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: spawned uWSGI worker 2 (pid: 80129, cores: 1) Jul 04 05:39:58.231350 dv-cinder-797333-5-iscsi devstack@keystone.service[80118]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 04 05:40:03.622494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 05:40:03.622494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: from cryptography.utils import int_from_bytes Jul 04 05:40:03.623281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 05:40:03.623281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: from cryptography.utils import int_from_bytes Jul 04 05:40:03.623705 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 05:40:03.623705 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: from cryptography.utils import int_from_bytes Jul 04 05:40:03.623815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 05:40:03.623815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: from cryptography.utils import int_from_bytes Jul 04 05:40:03.671372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 05:40:03.671557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 05:40:03.671634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 05:40:03.671712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 05:40:03.671779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] command line args: [] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 05:40:03.671846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] command line args: [] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 05:40:03.671997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 05:40:03.672076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 05:40:03.672185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ================================================================================ {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 05:40:03.672261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ================================================================================ {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 05:40:03.672454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] admin_token = **** {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.672589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] admin_token = **** {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.672672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] config_dir = [] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.672732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] config_dir = [] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.672862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.672948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.673009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] config_source = [] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.673074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] config_source = [] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.673206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] debug = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.673285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] debug = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.673351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.673652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.673746 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.673812 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.673876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.673934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.674114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.674204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.674376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.674535 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.674651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] list_limit = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.674756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] list_limit = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.674756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] log_config_append = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.674955 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] log_config_append = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.675056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.675160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.675288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] log_dir = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.675403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] log_dir = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.675517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] log_file = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.675621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] log_file = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.675621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.675824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.675948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676478 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.676992 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.677060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.677171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.677285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.677395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.677472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.677531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.677642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.677727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.677840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.677925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.678038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.678118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.678311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.678452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.678592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.678677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.679041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.679179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.679179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] publish_errors = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.679358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] publish_errors = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.679445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.679530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.679687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.679820 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.679820 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.680033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.680111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.680252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.680338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.680413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.680500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] standard_threads = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.680616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] standard_threads = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.680777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.680852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.680964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] use_journal = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] use_journal = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] use_json = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] use_json = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] use_stderr = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] use_stderr = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] use_syslog = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.681973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] use_syslog = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.682084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.682195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 05:40:03.682306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.682425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.682539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.682651 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.682769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.682888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.683000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.683120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.683240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.683391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.683512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.683626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.683737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.683856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.683972 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.684090 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.684203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.684320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.684432 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.684552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.684666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.684777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.684897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.685016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.685130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.685263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.685384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.685502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.685623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.685736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.685848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.686042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.686159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.686271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.686437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.686562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.686679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.686800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.686915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.687025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.687136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.687262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.687376 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.687502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.687616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.687728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.687840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.687952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.688063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.688184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.688298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.688408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.688518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.688638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.688750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.688871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.688984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.689095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.689212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.689323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.689442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.689554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.689667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.689778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.689890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.690000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.690119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.690232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.690349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.690466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.690578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.690689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.690809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.690930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.690930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.691139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.691266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.691381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.691571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.691693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.691805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.691917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.692028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.692138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.692258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.692371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.692481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.692600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.693067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.693261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.693388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.693507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.693620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.693731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.693850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.693962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.694084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.694198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.694309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.694420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.694524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.694625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.694730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.694826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.694922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.695016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.695117 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.695305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.695428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.695534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.695656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.695770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.695883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.695995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.696119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.696233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.696345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.696456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.696456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.696678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.696801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.696928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.697039 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.697161 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.697275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.697396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.697518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.697623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.697742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.697856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.697961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.698074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.698185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.698297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.698422 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.698537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.698640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.698751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.698863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.698975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.699097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.699211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.699336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.699449 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.699449 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.699687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.699687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.699899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.700011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.700122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.700122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.700341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.700455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.700572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.700684 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.700796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.700908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.701028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.701142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.701254 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.701367 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.701479 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.701590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.701711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.701825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.701937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.702055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.702175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.702289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.702415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.702529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.702642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.702760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.702873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.702995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.703186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.703366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.703483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.703602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.703715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.703828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.703951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.704074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.704204 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.704319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.704431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.704544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.704656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.704769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.704890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.705007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.705120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.705232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.705344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.705456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.705578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.705699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.705820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.706001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.706119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.706231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.706353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.706467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.706587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.706701 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.706814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.706933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.707055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.707169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] auth.external = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.707295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] auth.external = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.707420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.707535 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.707656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.707802 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.707932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.708053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.708167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] auth.password = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.708280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] auth.password = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.708393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] auth.token = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.708505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] auth.token = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.708626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.708771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.708888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.709001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.709121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.709244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.709367 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.709482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.709603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.709718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.709832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.709945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.710067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.710182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.710301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] credential.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.710423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] credential.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.710548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.710672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.710796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.710911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.711025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.711138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.711267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.711385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.711512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.711628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.711741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.711855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.711975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.712089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.712210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.712324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.712437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.712557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.712672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.712787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.712787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.713010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.713123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.713235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.713353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.713467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.713467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.713692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.713814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.713929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.714041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] federation.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.714155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] federation.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.714277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.714391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.714504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.714616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.714737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.714851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.714978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.715094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.715206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.715330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.715444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.715557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.715683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.715799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.715912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.716026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.716138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.716251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.716373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.716488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.716601 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.716713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.716826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.716948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.717136 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.717254 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.717368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.717481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.717594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.717716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.717831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.717944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.718058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.718171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.718284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.718405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.718520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.718633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.718746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.718859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.718972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.719093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.719209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.719370 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.719484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.719597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.719710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.719836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.719952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.720065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.720178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.720292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.720405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.720532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.720648 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.720763 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.720877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.720990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.721104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.721226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.721341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.721454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.721567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.721680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.721793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.721914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.722028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.722141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.722261 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.722376 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.722489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.722611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.722726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.722839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.722952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.723065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.723177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.723316 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.723452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.723567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.723681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.723794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.723907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.724029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.724152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.724267 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.724413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.724529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.724642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.724764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.724880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.724994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.725107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.725223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.725338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.725465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.725582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.725696 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.725809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.725921 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.726034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.726160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.726275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.726388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.726501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.726615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.726728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.726850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.726965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.727078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.727198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.727333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.727448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.727575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.727691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.727805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.727919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.728032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.728146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.728274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.728393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.728507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.728624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.728737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.728850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.728973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.728973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.729188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.729301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.729415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.729537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.729653 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.729767 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.729880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.729994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.730107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.730228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.730345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.730458 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.730573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.730686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.730800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.730922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.731037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.731150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.731319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.731436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.731559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.731683 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.731799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.731911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.732024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.732137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.732249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.732371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.732487 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.732600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.732713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.732827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.732940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.733063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.733178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.733292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.733405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.733518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.733631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.733753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.733869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.733982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.734095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.734208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.734329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.734329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.734552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.734665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.734778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.734890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.735004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.735125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.735239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.735391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.735504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.735618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.735731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.735857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.735974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.736087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.736200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.736314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.736426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.736548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.736663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.736776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.736889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.737003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.737116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.737237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.737352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.737465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] resource.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.737578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] resource.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.737692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.737805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.737927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.738042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.738156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.738269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.738382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.738494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.738621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.738736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.738849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.738962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.739075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.739189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.739404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.739522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.739635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.739749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.739862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] role.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.740000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] role.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.740128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] role.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.740244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] role.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.740359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.740472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.740589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.740703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.740825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.740941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.741056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.741169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.741284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.741399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.741526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.741642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.741758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.741871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.741985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.742107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.742223 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.742337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.742451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.742564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.742677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.742799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.742915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.743038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.743153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.743298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.743448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.743578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.743695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.743809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.743922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.744036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.744150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.744273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.744390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.744504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.744618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.744731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.744845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.744968 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.745083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.745197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.745311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.745426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.745539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.745661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.745777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.745891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.746004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.746118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.746232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.746354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.746471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.746585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.746699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.746813 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.746936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.747051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.747164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.747296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.747412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.747526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.747652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.747768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.747882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.747996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.748109 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] token.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.748223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] token.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.748345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.748460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.748576 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.748690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.748803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.748916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.749044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.749169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.749284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.749402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.749517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.749632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.749754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.749870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.749983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.750097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.750210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.750324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.750446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.750564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.750679 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.750793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.750898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.751012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.751135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.751305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.751422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.751546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.751660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.751774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.751901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.752017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.752131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.752244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.752357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.connection = **** {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.752480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.connection = **** {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.752608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.752733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.752848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.752970 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.753085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.753199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.753313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.753426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.753540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.753665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.753782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.753895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.754009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.754124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.754237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.754358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.754473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.754587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.754700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.754813 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.754927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.755049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.755164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.755291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.755406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.755539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.755685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.755812 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.755930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.756043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.756156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.756269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.756382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.756504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.756619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.756619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80129) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 05:40:03.756832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 05:40:03.756832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80128) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 05:40:03.784752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.784908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.785039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.785118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.785181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.785245 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.793529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.793626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.821883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.822101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.822225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.822388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.968797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:03.969006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.017620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.017829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.018340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.018580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.082541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.082719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.082843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.082948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.083070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.083151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.083273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.083365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.083488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.083621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.083777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.083988 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.108347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.108510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.108674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.108759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.108822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.108895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.108968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.109100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.109179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.109242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.109362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.109479 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.109556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.109682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.120692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.120865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.133274 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.133543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.133666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.133796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.133880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.133986 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.134076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.134224 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.134349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.134480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.134563 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.134641 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.134717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80129) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.134801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80128) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 05:40:04.151873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.152078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 05:40:04.152406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.152532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.152697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 05:40:04.152783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 05:40:04.152847 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.152919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.153092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.153217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 05:40:04.153368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.153452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.153558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.153648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.153851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.154106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.164362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.164674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.164996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.165122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.165254 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.165360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.165489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.165769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.173405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.173993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.176660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.176936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.177204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.177327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.177467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.177608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.177761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.177850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.178044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.178170 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.178320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.178404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.178554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.178680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.178817 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.178899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.179068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.179194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.179322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.179412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.179679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.179939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.206198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.206815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.207807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.208078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.208456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.208736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.213462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.213732 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.214017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.214143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.214302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.214673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.221336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.221612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.221948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.222226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.226029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.226337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.226611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.226737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.226889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.226981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.227229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.227385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.227530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.227610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.227788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.227912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.228050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.228173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.228254 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.228551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.246596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.246868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.247073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.247356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.251487 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.251985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.255076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.255386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.255638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.255918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.260610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.261035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.264197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.264665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.267827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.268115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.268321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.268467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.268606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.268689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.268903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.269031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.269168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.269249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.269459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.269738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.288241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.288524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.288782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.288905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.289067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.289151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.289342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.289608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.296935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.297482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.298631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.298901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.299195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.299493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.302074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.302347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.302701 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.302825 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.302965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.303047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.303281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.303405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.303542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.303814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.316453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.316725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.316983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.317125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.317262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.317353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.317467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.317548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.318123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.318446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.319437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.319726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.320003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.320299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.337681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.338157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.340879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.341444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.344178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.344775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.345861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.346498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.347514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.347787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.348067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.348192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.348435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.348723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.355514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.356077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.357107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.357682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.360303 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.360581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.360905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.361032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.361172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.361276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.361568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.361839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.370401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.370686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.370942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.371089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.371206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.371535 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.378716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80129) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.378993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.379277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.379401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80128) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 05:40:04.379539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.379620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.379809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.379937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.380175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.380301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.380441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.380522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.380764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.380904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.380989 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.381091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.381294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.381478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.381567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.381643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.381808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.381948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.382044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.382160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.382320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80129) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.382448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.382704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.382984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80128) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 05:40:04.420611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55628d84e9c0 pid: 80128 (default app) Jul 04 05:40:04.421580 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55628d84e9c0 pid: 80129 (default app) Jul 04 05:40:04.424072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab9189-8adf-44c3-8703-2dd177fe2f1d None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:04.424363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab9189-8adf-44c3-8703-2dd177fe2f1d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:04.424647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab9189-8adf-44c3-8703-2dd177fe2f1d None None] PATH_INFO: `/v3` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:04.428999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Mon Jul 4 05:40:04 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:12.102681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943722dd-69bd-433e-a865-9c05b7d0541e None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:12.103639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943722dd-69bd-433e-a865-9c05b7d0541e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:12.103639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943722dd-69bd-433e-a865-9c05b7d0541e None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:12.110731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:12 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:12.117039 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:12.117802 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:12.117959 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:12.192737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: INFO dbcounter [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] Registered counter for database keystone Jul 04 05:40:12.217686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG oslo_db.sqlalchemy.engines [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80128) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 05:40:12.229515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writer thread running {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 05:40:12.416962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG passlib.handlers.bcrypt [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80128) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 05:40:12.417326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG passlib.handlers.bcrypt [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80128) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 05:40:12.472607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:12.474224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:12.608605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 04 05:40:12.807212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943722dd-69bd-433e-a865-9c05b7d0541e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:12.882649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: INFO dbcounter [None req-943722dd-69bd-433e-a865-9c05b7d0541e None None] Registered counter for database keystone Jul 04 05:40:12.904420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG oslo_db.sqlalchemy.engines [None req-943722dd-69bd-433e-a865-9c05b7d0541e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80129) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 05:40:12.915516 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writer thread running {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 05:40:13.035388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943722dd-69bd-433e-a865-9c05b7d0541e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:13.041270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345d8e4-e106-4b54-bdb0-8e96d5e56c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:13.041948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345d8e4-e106-4b54-bdb0-8e96d5e56c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:13.042236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345d8e4-e106-4b54-bdb0-8e96d5e56c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:13.042510 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345d8e4-e106-4b54-bdb0-8e96d5e56c6d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:13.051969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345d8e4-e106-4b54-bdb0-8e96d5e56c6d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:13.104005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4345d8e4-e106-4b54-bdb0-8e96d5e56c6d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 05:40:13.104572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345d8e4-e106-4b54-bdb0-8e96d5e56c6d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:13.114755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Jul 4 05:40:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 05:40:13.122183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:13.145241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772548a-9014-4378-be1a-2f6e5383a0ac None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:13.150434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0e8cf-dab5-4404-83c6-c20efdb854f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:13.151648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0e8cf-dab5-4404-83c6-c20efdb854f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:13.151936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0e8cf-dab5-4404-83c6-c20efdb854f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:13.152216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0e8cf-dab5-4404-83c6-c20efdb854f0 admin admin] PATH_INFO: `/v3/projects/8a4d1bfa73f141cd90a91a1d9547a701` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:13.156430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0e8cf-dab5-4404-83c6-c20efdb854f0 admin admin] RBAC: Authorizing `identity:get_project(project_id=8a4d1bfa73f141cd90a91a1d9547a701)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:13.206619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4ea0e8cf-dab5-4404-83c6-c20efdb854f0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 05:40:13.206953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0e8cf-dab5-4404-83c6-c20efdb854f0 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:13.211664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Jul 4 05:40:13 2022] GET /identity/v3/projects/8a4d1bfa73f141cd90a91a1d9547a701 => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:15.211309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7570352-067f-4d8f-80fa-ddec8e55865a None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:15.212200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7570352-067f-4d8f-80fa-ddec8e55865a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:15.212321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7570352-067f-4d8f-80fa-ddec8e55865a None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:15.219111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:15.224888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd731e9f-4c77-49a9-b7c2-47523adf92a0 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:15.225343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd731e9f-4c77-49a9-b7c2-47523adf92a0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:15.225432 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd731e9f-4c77-49a9-b7c2-47523adf92a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:15.246159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-cd731e9f-4c77-49a9-b7c2-47523adf92a0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:15.247734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cd731e9f-4c77-49a9-b7c2-47523adf92a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:15.256271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 04 05:40:15.279319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7570352-067f-4d8f-80fa-ddec8e55865a None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:15.301233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7570352-067f-4d8f-80fa-ddec8e55865a None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:15.308018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b07e3-00e4-47a4-9673-4c13ccde7bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:15.308668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b07e3-00e4-47a4-9673-4c13ccde7bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:15.308948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b07e3-00e4-47a4-9673-4c13ccde7bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:15.309217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b07e3-00e4-47a4-9673-4c13ccde7bb1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:15.317753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b07e3-00e4-47a4-9673-4c13ccde7bb1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:15.319930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b07e3-00e4-47a4-9673-4c13ccde7bb1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:15.329243 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Jul 4 05:40:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 05:40:15.337408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd731e9f-4c77-49a9-b7c2-47523adf92a0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:15.352354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd731e9f-4c77-49a9-b7c2-47523adf92a0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:15.359738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74324e1a-2dc9-437e-8884-284259e3de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:15.360621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74324e1a-2dc9-437e-8884-284259e3de9f admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:15.360899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74324e1a-2dc9-437e-8884-284259e3de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:15.361175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74324e1a-2dc9-437e-8884-284259e3de9f admin admin] PATH_INFO: `/v3/users/825e62d400e54cc0868912432c257354` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:15.362620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74324e1a-2dc9-437e-8884-284259e3de9f admin admin] RBAC: Authorizing `identity:get_user(user_id=825e62d400e54cc0868912432c257354)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:15.364512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74324e1a-2dc9-437e-8884-284259e3de9f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:15.366671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Mon Jul 4 05:40:15 2022] GET /identity/v3/users/825e62d400e54cc0868912432c257354 => generated 247 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:17.464985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7433e-94da-41e9-b45e-f6ae47fb1ca6 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:17.465566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7433e-94da-41e9-b45e-f6ae47fb1ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:17.465566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7433e-94da-41e9-b45e-f6ae47fb1ca6 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:17.466952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:17.471073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69cb2c-2560-45cd-97b5-01e354a10e24 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:17.471558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69cb2c-2560-45cd-97b5-01e354a10e24 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:17.471650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69cb2c-2560-45cd-97b5-01e354a10e24 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:17.472027 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b9de3-92a2-45f2-aa92-3f07c9870218 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:17.472322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b9de3-92a2-45f2-aa92-3f07c9870218 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:17.472595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b9de3-92a2-45f2-aa92-3f07c9870218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:17.473029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:17.481676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831ed2a-5e00-447a-84ce-246db7382eb8 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:17.481965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831ed2a-5e00-447a-84ce-246db7382eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:17.482413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831ed2a-5e00-447a-84ce-246db7382eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:17.502643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-8831ed2a-5e00-447a-84ce-246db7382eb8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:17.503953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8831ed2a-5e00-447a-84ce-246db7382eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:17.516881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG passlib.handlers.bcrypt [None req-951b9de3-92a2-45f2-aa92-3f07c9870218 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80129) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 05:40:17.517308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG passlib.handlers.bcrypt [None req-951b9de3-92a2-45f2-aa92-3f07c9870218 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80129) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 05:40:17.562349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-951b9de3-92a2-45f2-aa92-3f07c9870218 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:17.564001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-951b9de3-92a2-45f2-aa92-3f07c9870218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:17.593505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 7/12] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:17.615834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831ed2a-5e00-447a-84ce-246db7382eb8 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:17.639794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:17.666422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 6/13] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:17.689942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b9de3-92a2-45f2-aa92-3f07c9870218 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:17.721143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831ed2a-5e00-447a-84ce-246db7382eb8 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:17.754406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4bbfb-8a52-4b93-8433-bea6364d27ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:17.755061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4bbfb-8a52-4b93-8433-bea6364d27ae None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:17.755368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4bbfb-8a52-4b93-8433-bea6364d27ae None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:17.755638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4bbfb-8a52-4b93-8433-bea6364d27ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:17.786661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b9de3-92a2-45f2-aa92-3f07c9870218 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:17.788937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:17.823670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4bbfb-8a52-4b93-8433-bea6364d27ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:17.823980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691d58f-354f-4dab-8402-35946372b254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:17.825041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691d58f-354f-4dab-8402-35946372b254 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:17.825177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4bbfb-8a52-4b93-8433-bea6364d27ae None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:17.825323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691d58f-354f-4dab-8402-35946372b254 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:17.825596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691d58f-354f-4dab-8402-35946372b254 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:17.837024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691d58f-354f-4dab-8402-35946372b254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:17.847594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-9691d58f-354f-4dab-8402-35946372b254 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:17.848376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:17.854864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:40:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:40:17.855781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691d58f-354f-4dab-8402-35946372b254 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:17.925979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 8/15] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 05:40:17.930456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4bbfb-8a52-4b93-8433-bea6364d27ae None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:17.954036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691d58f-354f-4dab-8402-35946372b254 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:17.986731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6590a38-0372-467f-bde3-7a78ba833f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:17.987427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6590a38-0372-467f-bde3-7a78ba833f0d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:17.987706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6590a38-0372-467f-bde3-7a78ba833f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:17.987976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6590a38-0372-467f-bde3-7a78ba833f0d None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:17.988725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6590a38-0372-467f-bde3-7a78ba833f0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:17.989025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6590a38-0372-467f-bde3-7a78ba833f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:17.990500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6590a38-0372-467f-bde3-7a78ba833f0d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:18.002395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 8/16] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:18.008101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6590a38-0372-467f-bde3-7a78ba833f0d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:18.025477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4bbfb-8a52-4b93-8433-bea6364d27ae None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:18.034675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:18.049110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:18.061217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c38fa-9863-492d-b59a-be7c9e695d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:18.062022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c38fa-9863-492d-b59a-be7c9e695d9b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:18.062306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c38fa-9863-492d-b59a-be7c9e695d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:18.062584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c38fa-9863-492d-b59a-be7c9e695d9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:18.073076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c38fa-9863-492d-b59a-be7c9e695d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:18.074570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c38fa-9863-492d-b59a-be7c9e695d9b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-268c38fa-9863-492d-b59a-be7c9e695d9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:40:18.095140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:18.096130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:18.096925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:40:18.096925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:40:18.096925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:40:18.096925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:18.096925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:40:18.102214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c38fa-9863-492d-b59a-be7c9e695d9b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:18.109102 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6590a38-0372-467f-bde3-7a78ba833f0d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:18.143730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06fda9-5048-427a-ab7b-461c30c252b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:18.144619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06fda9-5048-427a-ab7b-461c30c252b8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:18.144905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06fda9-5048-427a-ab7b-461c30c252b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:18.145180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06fda9-5048-427a-ab7b-461c30c252b8 None admin] PATH_INFO: `/v3/users/825e62d400e54cc0868912432c257354` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:18.146799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06fda9-5048-427a-ab7b-461c30c252b8 None admin] RBAC: Authorizing `identity:get_user(user_id=825e62d400e54cc0868912432c257354)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:18.148320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06fda9-5048-427a-ab7b-461c30c252b8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:18.149959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:18 2022] GET /identity/v3/users/825e62d400e54cc0868912432c257354 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:18.155557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06fda9-5048-427a-ab7b-461c30c252b8 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:18.204490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c38fa-9863-492d-b59a-be7c9e695d9b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:18.240113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fe250-500c-4084-88d8-2060b9b5f2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:18.240753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fe250-500c-4084-88d8-2060b9b5f2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:18.241037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fe250-500c-4084-88d8-2060b9b5f2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:18.241313 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fe250-500c-4084-88d8-2060b9b5f2e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:18.242051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fe250-500c-4084-88d8-2060b9b5f2e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:18.242353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fe250-500c-4084-88d8-2060b9b5f2e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:18.243901 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fe250-500c-4084-88d8-2060b9b5f2e6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:18.247302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:18.254222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06fda9-5048-427a-ab7b-461c30c252b8 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:18.261334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:18.289774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0e41e-8d82-44dc-aab9-72fe44734bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:18.290601 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0e41e-8d82-44dc-aab9-72fe44734bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:18.290886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0e41e-8d82-44dc-aab9-72fe44734bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:18.291164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0e41e-8d82-44dc-aab9-72fe44734bd9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:18.292473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0e41e-8d82-44dc-aab9-72fe44734bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:18.294002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0e41e-8d82-44dc-aab9-72fe44734bd9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:18.295706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:18.302130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fe250-500c-4084-88d8-2060b9b5f2e6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:18.340359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:18.412794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fe250-500c-4084-88d8-2060b9b5f2e6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:18.452456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc6cb3-c5d1-48bf-90e9-735b9bd486d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:18.453135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc6cb3-c5d1-48bf-90e9-735b9bd486d8 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:18.453420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc6cb3-c5d1-48bf-90e9-735b9bd486d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:18.453696 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc6cb3-c5d1-48bf-90e9-735b9bd486d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:18.454539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc6cb3-c5d1-48bf-90e9-735b9bd486d8 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.domain.id=default, user.id=825e62d400e54cc0868912432c257354) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:18.454852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc6cb3-c5d1-48bf-90e9-735b9bd486d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:18.456374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc6cb3-c5d1-48bf-90e9-735b9bd486d8 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:18.463515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 11/21] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Jul 4 05:40:18 2022] GET /identity/v3/role_assignments?user.id=825e62d400e54cc0868912432c257354&scope.domain.id=default&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:20.867322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19bcf2-8fa6-4f23-a41a-1b38aeb1331e None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:20.868341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19bcf2-8fa6-4f23-a41a-1b38aeb1331e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:20.868922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19bcf2-8fa6-4f23-a41a-1b38aeb1331e None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:20.870650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:20.876703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279cbd0-f8ca-41e2-87e7-511764c7946d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:20.877574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279cbd0-f8ca-41e2-87e7-511764c7946d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:20.878044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279cbd0-f8ca-41e2-87e7-511764c7946d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:20.904251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c279cbd0-f8ca-41e2-87e7-511764c7946d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:20.905869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c279cbd0-f8ca-41e2-87e7-511764c7946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:20.996630 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 12/23] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:21.039332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19bcf2-8fa6-4f23-a41a-1b38aeb1331e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:21.121833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53ffce-4f63-4cdd-bfb1-8b76687181a9 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.122456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53ffce-4f63-4cdd-bfb1-8b76687181a9 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.122923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53ffce-4f63-4cdd-bfb1-8b76687181a9 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.124518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:40:21.129546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa867c46-e238-4c1f-a71b-f68430e0be88 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.130047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa867c46-e238-4c1f-a71b-f68430e0be88 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.130531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa867c46-e238-4c1f-a71b-f68430e0be88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.157877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-fa867c46-e238-4c1f-a71b-f68430e0be88 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:21.159877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-fa867c46-e238-4c1f-a71b-f68430e0be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:21.197807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19bcf2-8fa6-4f23-a41a-1b38aeb1331e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:21.207586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:21.260326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da1719-97f2-4716-82d4-736a9fa11707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:21.261146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da1719-97f2-4716-82d4-736a9fa11707 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.261428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da1719-97f2-4716-82d4-736a9fa11707 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.261706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da1719-97f2-4716-82d4-736a9fa11707 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.284589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da1719-97f2-4716-82d4-736a9fa11707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-d8da1719-97f2-4716-82d4-736a9fa11707 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:21.293468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:21.294345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:21.295108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:21.297344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 12/25] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:40:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:21.298690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 14/26] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:21.299688 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99de87-4585-4b68-8942-26072a23cee8 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.300160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3939edb-e48c-41e6-8dde-469daf9b7722 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.300409 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99de87-4585-4b68-8942-26072a23cee8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.300502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3939edb-e48c-41e6-8dde-469daf9b7722 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.300718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3939edb-e48c-41e6-8dde-469daf9b7722 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.301147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99de87-4585-4b68-8942-26072a23cee8 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.301996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 15/27] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:21.302844 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 13/28] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:21.308409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35462448-afde-41e0-b78a-c369cf437d39 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.308958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35462448-afde-41e0-b78a-c369cf437d39 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.309428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35462448-afde-41e0-b78a-c369cf437d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.311056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102541-8a8f-4a36-87ba-1f872c1990cd None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.311364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102541-8a8f-4a36-87ba-1f872c1990cd None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.311640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102541-8a8f-4a36-87ba-1f872c1990cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.335395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d6102541-8a8f-4a36-87ba-1f872c1990cd None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:21.335756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-35462448-afde-41e0-b78a-c369cf437d39 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:21.336590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d6102541-8a8f-4a36-87ba-1f872c1990cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:21.336922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-35462448-afde-41e0-b78a-c369cf437d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:21.354964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:21.414637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 14/29] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:21.415440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6102541-8a8f-4a36-87ba-1f872c1990cd None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:21.426900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 16/30] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:21.427764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35462448-afde-41e0-b78a-c369cf437d39 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:21.554562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6102541-8a8f-4a36-87ba-1f872c1990cd None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:21.569075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:21.585347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35462448-afde-41e0-b78a-c369cf437d39 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:21.593299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2497d81-7abf-4b34-ad22-b82863d58dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:21.594204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2497d81-7abf-4b34-ad22-b82863d58dcd None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.594663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2497d81-7abf-4b34-ad22-b82863d58dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.595130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2497d81-7abf-4b34-ad22-b82863d58dcd None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.596046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2497d81-7abf-4b34-ad22-b82863d58dcd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:21.596507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2497d81-7abf-4b34-ad22-b82863d58dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:21.598162 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2497d81-7abf-4b34-ad22-b82863d58dcd None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:21.607961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 15/31] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:21.608937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2497d81-7abf-4b34-ad22-b82863d58dcd None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:21.625760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:21.629138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5751e8-994b-41b2-b79a-c68a650b92f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:21.630017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5751e8-994b-41b2-b79a-c68a650b92f4 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.630299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5751e8-994b-41b2-b79a-c68a650b92f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.630627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5751e8-994b-41b2-b79a-c68a650b92f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.632213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5751e8-994b-41b2-b79a-c68a650b92f4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:21.636372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5751e8-994b-41b2-b79a-c68a650b92f4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:21.721549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2497d81-7abf-4b34-ad22-b82863d58dcd None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:21.756669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab5e3f-7f88-4daa-ae9f-877e9a1ca6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:21.757613 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5e3f-7f88-4daa-ae9f-877e9a1ca6fb None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.758030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5e3f-7f88-4daa-ae9f-877e9a1ca6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.758436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5e3f-7f88-4daa-ae9f-877e9a1ca6fb None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.759573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab5e3f-7f88-4daa-ae9f-877e9a1ca6fb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:21.761217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab5e3f-7f88-4daa-ae9f-877e9a1ca6fb None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:21.769202 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 17/32] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jul 4 05:40:21 2022] POST /identity/v3/roles => generated 219 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:21.770590 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5751e8-994b-41b2-b79a-c68a650b92f4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:21.889445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5751e8-994b-41b2-b79a-c68a650b92f4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:21.916860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 16/33] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jul 4 05:40:21 2022] POST /identity/v3/roles => generated 225 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:21.918169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab5e3f-7f88-4daa-ae9f-877e9a1ca6fb None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:21.931782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4476d-d508-41af-a8ea-6c6318f995f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:21.932857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4476d-d508-41af-a8ea-6c6318f995f9 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.933284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4476d-d508-41af-a8ea-6c6318f995f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.934368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4476d-d508-41af-a8ea-6c6318f995f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.950140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4476d-d508-41af-a8ea-6c6318f995f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:21.952179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4476d-d508-41af-a8ea-6c6318f995f9 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:21.964614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-a7f4476d-d508-41af-a8ea-6c6318f995f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:40:21.975910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:21.976854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:21.977657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:40:21.977657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:40:21.977657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:40:21.977657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:21.979518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 18/34] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:21.981078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c266ff-c4d1-4555-9ba6-fb06caededd5 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:21.981407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c266ff-c4d1-4555-9ba6-fb06caededd5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:21.981697 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c266ff-c4d1-4555-9ba6-fb06caededd5 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:21.987194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 19/35] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:21.988121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c266ff-c4d1-4555-9ba6-fb06caededd5 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:22.018690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:22.045795 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab5e3f-7f88-4daa-ae9f-877e9a1ca6fb None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:22.088473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf8086-3188-4402-bdd6-600ac507ddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:22.089568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf8086-3188-4402-bdd6-600ac507ddbb None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.090073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf8086-3188-4402-bdd6-600ac507ddbb None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.090423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf8086-3188-4402-bdd6-600ac507ddbb None admin] PATH_INFO: `/v3/users/825e62d400e54cc0868912432c257354` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.092015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf8086-3188-4402-bdd6-600ac507ddbb None admin] RBAC: Authorizing `identity:get_user(user_id=825e62d400e54cc0868912432c257354)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:22.093745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf8086-3188-4402-bdd6-600ac507ddbb None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:22.095643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 17/36] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:21 2022] GET /identity/v3/users/825e62d400e54cc0868912432c257354 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:22.101978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5a8b2-3258-4234-a8e8-9f1f0dbd17bb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.102274 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5a8b2-3258-4234-a8e8-9f1f0dbd17bb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.102559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5a8b2-3258-4234-a8e8-9f1f0dbd17bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.118827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c266ff-c4d1-4555-9ba6-fb06caededd5 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:22.121019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-3df5a8b2-3258-4234-a8e8-9f1f0dbd17bb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:22.122558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3df5a8b2-3258-4234-a8e8-9f1f0dbd17bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:22.156110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b8cde-88aa-4084-945b-2a295fc06127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:22.156793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8cde-88aa-4084-945b-2a295fc06127 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.157502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8cde-88aa-4084-945b-2a295fc06127 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.157502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8cde-88aa-4084-945b-2a295fc06127 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.158076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b8cde-88aa-4084-945b-2a295fc06127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:22.158383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b8cde-88aa-4084-945b-2a295fc06127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:22.159839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b8cde-88aa-4084-945b-2a295fc06127 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:22.173845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 20/37] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:22.175301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7170b-0a6e-4c96-ac5d-1ce7ee772b27 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.175597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7170b-0a6e-4c96-ac5d-1ce7ee772b27 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.175872 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7170b-0a6e-4c96-ac5d-1ce7ee772b27 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.177095 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 21/38] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:22.177781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7170b-0a6e-4c96-ac5d-1ce7ee772b27 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:22.199334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 18/39] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:22.200176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5a8b2-3258-4234-a8e8-9f1f0dbd17bb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:22.238847 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:22.279619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:22.281098 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7170b-0a6e-4c96-ac5d-1ce7ee772b27 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:22.309987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5a8b2-3258-4234-a8e8-9f1f0dbd17bb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:22.316611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a81e-d7d3-4176-ba6b-cd56778bda17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:22.317801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a81e-d7d3-4176-ba6b-cd56778bda17 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.318083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a81e-d7d3-4176-ba6b-cd56778bda17 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.318685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a81e-d7d3-4176-ba6b-cd56778bda17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.320336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8a81e-d7d3-4176-ba6b-cd56778bda17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:22.322126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8a81e-d7d3-4176-ba6b-cd56778bda17 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:22.323805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 22/40] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:22.328572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29932020-b460-4c06-88f6-ddb4fcd33b64 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.328572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29932020-b460-4c06-88f6-ddb4fcd33b64 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.328572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29932020-b460-4c06-88f6-ddb4fcd33b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.342647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-29932020-b460-4c06-88f6-ddb4fcd33b64 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:22.344136 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116c88d-e7f4-4cca-bceb-b0b627c07054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:22.344270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-29932020-b460-4c06-88f6-ddb4fcd33b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:22.344816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116c88d-e7f4-4cca-bceb-b0b627c07054 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.345095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116c88d-e7f4-4cca-bceb-b0b627c07054 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.345370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116c88d-e7f4-4cca-bceb-b0b627c07054 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.346095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116c88d-e7f4-4cca-bceb-b0b627c07054 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:22.347586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116c88d-e7f4-4cca-bceb-b0b627c07054 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:22.384173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 19/41] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:40:22 2022] POST /identity/v3/projects => generated 301 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:22.384799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116c88d-e7f4-4cca-bceb-b0b627c07054 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:22.520071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116c88d-e7f4-4cca-bceb-b0b627c07054 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:22.528865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 23/42] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:22.529733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29932020-b460-4c06-88f6-ddb4fcd33b64 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:22.560322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:22.575625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c2a54-5369-490e-99fd-521d42e82516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:22.576323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c2a54-5369-490e-99fd-521d42e82516 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.576612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c2a54-5369-490e-99fd-521d42e82516 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.576894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c2a54-5369-490e-99fd-521d42e82516 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.577669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c2a54-5369-490e-99fd-521d42e82516 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:22.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c2a54-5369-490e-99fd-521d42e82516 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:22.595452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 20/43] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jul 4 05:40:22 2022] POST /identity/v3/roles => generated 223 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:22.596075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c2a54-5369-490e-99fd-521d42e82516 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:22.603974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:22.657761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29932020-b460-4c06-88f6-ddb4fcd33b64 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:22.700231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e691b4-c9d5-4e79-a7a5-1e3b80016a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:22.701205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e691b4-c9d5-4e79-a7a5-1e3b80016a67 None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.701492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e691b4-c9d5-4e79-a7a5-1e3b80016a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.701821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e691b4-c9d5-4e79-a7a5-1e3b80016a67 None admin] PATH_INFO: `/v3/domains/default/users/825e62d400e54cc0868912432c257354/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.704381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e691b4-c9d5-4e79-a7a5-1e3b80016a67 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=825e62d400e54cc0868912432c257354, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:22.705936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e691b4-c9d5-4e79-a7a5-1e3b80016a67 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:22.727298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 24/44] 10.222.0.42 () {58 vars in 1361 bytes} [Mon Jul 4 05:40:22 2022] PUT /identity/v3/domains/default/users/825e62d400e54cc0868912432c257354/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:40:22.727534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c2a54-5369-490e-99fd-521d42e82516 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:22.774195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e117dd-66e3-4f55-95b9-c707cc647619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:22.775049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e117dd-66e3-4f55-95b9-c707cc647619 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.775262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e117dd-66e3-4f55-95b9-c707cc647619 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.775607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e117dd-66e3-4f55-95b9-c707cc647619 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.776995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e117dd-66e3-4f55-95b9-c707cc647619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:22.778575 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e117dd-66e3-4f55-95b9-c707cc647619 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:22.780672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 21/45] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:22.786686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e691b4-c9d5-4e79-a7a5-1e3b80016a67 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:22.902761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e691b4-c9d5-4e79-a7a5-1e3b80016a67 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:22.919907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:22.937875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9800fb-1ffd-43dd-aba9-e95c4eefa925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:22.938553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9800fb-1ffd-43dd-aba9-e95c4eefa925 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:22.938839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9800fb-1ffd-43dd-aba9-e95c4eefa925 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:22.939113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9800fb-1ffd-43dd-aba9-e95c4eefa925 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:22.939873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9800fb-1ffd-43dd-aba9-e95c4eefa925 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:22.941328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9800fb-1ffd-43dd-aba9-e95c4eefa925 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:22.971096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 25/46] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:40:22 2022] POST /identity/v3/projects => generated 312 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:24.730539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc6355-48f9-43cc-b40e-2c64f7195615 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:24.731111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc6355-48f9-43cc-b40e-2c64f7195615 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:24.731111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc6355-48f9-43cc-b40e-2c64f7195615 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:24.732478 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 22/47] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:24.737907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4412a-e759-4552-8bb4-0dfd4e7be6cb None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:24.738307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4412a-e759-4552-8bb4-0dfd4e7be6cb None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:24.738506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4412a-e759-4552-8bb4-0dfd4e7be6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:24.756073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ecc4412a-e759-4552-8bb4-0dfd4e7be6cb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:24.757423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ecc4412a-e759-4552-8bb4-0dfd4e7be6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:24.828890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 26/48] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:24.852150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc6355-48f9-43cc-b40e-2c64f7195615 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:24.910355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:24.960615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc6355-48f9-43cc-b40e-2c64f7195615 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:24.982274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeac01f-fe18-43e2-9d57-d6c0fa940546 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:24.982571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeac01f-fe18-43e2-9d57-d6c0fa940546 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:24.982854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeac01f-fe18-43e2-9d57-d6c0fa940546 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:24.984262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 27/49] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:40:24.989732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ff0f7-9ea9-4cb3-9be4-71f3cd0de16b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:24.990020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ff0f7-9ea9-4cb3-9be4-71f3cd0de16b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:24.990303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ff0f7-9ea9-4cb3-9be4-71f3cd0de16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:24.995549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4edcca-f0d9-4d86-ac75-6bfb955a8634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:24.996358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4edcca-f0d9-4d86-ac75-6bfb955a8634 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:24.996640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4edcca-f0d9-4d86-ac75-6bfb955a8634 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:24.996916 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4edcca-f0d9-4d86-ac75-6bfb955a8634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:25.006886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4edcca-f0d9-4d86-ac75-6bfb955a8634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:25.007302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-248ff0f7-9ea9-4cb3-9be4-71f3cd0de16b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:25.008524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-248ff0f7-9ea9-4cb3-9be4-71f3cd0de16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-6a4edcca-f0d9-4d86-ac75-6bfb955a8634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:25.014537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:25.015589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:25.016337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 23/50] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:40:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:25.021507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4edcca-f0d9-4d86-ac75-6bfb955a8634 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:25.077422 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 28/51] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:25.100345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ff0f7-9ea9-4cb3-9be4-71f3cd0de16b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:25.122960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4edcca-f0d9-4d86-ac75-6bfb955a8634 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:25.129310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:25.136940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:25.158245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c8327-bc7a-47e1-8a94-2412f738b10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:25.158922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c8327-bc7a-47e1-8a94-2412f738b10e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:25.159210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c8327-bc7a-47e1-8a94-2412f738b10e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:25.159512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c8327-bc7a-47e1-8a94-2412f738b10e None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:25.160274 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c8327-bc7a-47e1-8a94-2412f738b10e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:25.160584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c8327-bc7a-47e1-8a94-2412f738b10e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:25.162015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c8327-bc7a-47e1-8a94-2412f738b10e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:25.170169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 24/52] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:25.175805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c8327-bc7a-47e1-8a94-2412f738b10e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:25.202311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ff0f7-9ea9-4cb3-9be4-71f3cd0de16b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:25.235997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6786e15-966e-4039-869c-9a79bd6dc219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:25.236809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6786e15-966e-4039-869c-9a79bd6dc219 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:25.237100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6786e15-966e-4039-869c-9a79bd6dc219 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:25.237381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6786e15-966e-4039-869c-9a79bd6dc219 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:25.238570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6786e15-966e-4039-869c-9a79bd6dc219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:25.240092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6786e15-966e-4039-869c-9a79bd6dc219 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:25.241808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 29/53] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:25.247376 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6786e15-966e-4039-869c-9a79bd6dc219 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:25.278331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c8327-bc7a-47e1-8a94-2412f738b10e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:25.311702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc571f0a-d95c-45a1-ac5f-8f2989b0e1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:25.312595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc571f0a-d95c-45a1-ac5f-8f2989b0e1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:25.312883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc571f0a-d95c-45a1-ac5f-8f2989b0e1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:25.313176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc571f0a-d95c-45a1-ac5f-8f2989b0e1c9 None admin] PATH_INFO: `/v3/users/825e62d400e54cc0868912432c257354` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:25.314369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc571f0a-d95c-45a1-ac5f-8f2989b0e1c9 None admin] RBAC: Authorizing `identity:get_user(user_id=825e62d400e54cc0868912432c257354)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:25.315887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc571f0a-d95c-45a1-ac5f-8f2989b0e1c9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:25.317570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 25/54] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:25 2022] GET /identity/v3/users/825e62d400e54cc0868912432c257354 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:25.323618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc571f0a-d95c-45a1-ac5f-8f2989b0e1c9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:25.343914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:25.348022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6786e15-966e-4039-869c-9a79bd6dc219 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:25.365537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:25.386550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518a5c9-56e0-49da-8bfd-5a00a3905e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:25.387236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518a5c9-56e0-49da-8bfd-5a00a3905e46 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:25.387553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518a5c9-56e0-49da-8bfd-5a00a3905e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:25.387831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518a5c9-56e0-49da-8bfd-5a00a3905e46 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:25.388560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518a5c9-56e0-49da-8bfd-5a00a3905e46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:25.390976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518a5c9-56e0-49da-8bfd-5a00a3905e46 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:25.418324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 30/55] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Jul 4 05:40:25 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:40:25.425040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc571f0a-d95c-45a1-ac5f-8f2989b0e1c9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:25.467013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d7e99-817f-4fd2-ae4b-3afa9ca6ab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:25.467859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d7e99-817f-4fd2-ae4b-3afa9ca6ab6d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:25.468139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d7e99-817f-4fd2-ae4b-3afa9ca6ab6d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:25.468406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d7e99-817f-4fd2-ae4b-3afa9ca6ab6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:25.469671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d7e99-817f-4fd2-ae4b-3afa9ca6ab6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:25.471163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d7e99-817f-4fd2-ae4b-3afa9ca6ab6d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:25.472879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 26/56] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:25.479456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518a5c9-56e0-49da-8bfd-5a00a3905e46 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:25.592840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518a5c9-56e0-49da-8bfd-5a00a3905e46 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:25.626676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bfc1f-4849-4eaa-a130-0b7a633e7933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:25.627315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bfc1f-4849-4eaa-a130-0b7a633e7933 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:25.627574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bfc1f-4849-4eaa-a130-0b7a633e7933 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:25.627854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bfc1f-4849-4eaa-a130-0b7a633e7933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:25.628704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bfc1f-4849-4eaa-a130-0b7a633e7933 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.domain.id=default, user.id=825e62d400e54cc0868912432c257354) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:25.629017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bfc1f-4849-4eaa-a130-0b7a633e7933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:25.630489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bfc1f-4849-4eaa-a130-0b7a633e7933 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:25.636660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 31/57] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Jul 4 05:40:25 2022] GET /identity/v3/role_assignments?user.id=825e62d400e54cc0868912432c257354&scope.domain.id=default&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:27.471087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067582f-275c-4e21-8fbf-d00ac9429398 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:27.471649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067582f-275c-4e21-8fbf-d00ac9429398 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:27.471649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067582f-275c-4e21-8fbf-d00ac9429398 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:27.473115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 27/58] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:27.479364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf3389-17ee-4837-8d12-7a388e539270 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:27.479778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf3389-17ee-4837-8d12-7a388e539270 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:27.479931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf3389-17ee-4837-8d12-7a388e539270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:27.501494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-bbdf3389-17ee-4837-8d12-7a388e539270 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:27.502728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-bbdf3389-17ee-4837-8d12-7a388e539270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:27.512783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:27.575947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 32/59] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:27.600737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067582f-275c-4e21-8fbf-d00ac9429398 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:27.829289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067582f-275c-4e21-8fbf-d00ac9429398 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:27.839929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:27.867916 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd97f3c-abe5-4629-92d0-b951903d1502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:27.868904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd97f3c-abe5-4629-92d0-b951903d1502 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:27.869272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd97f3c-abe5-4629-92d0-b951903d1502 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:27.869629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd97f3c-abe5-4629-92d0-b951903d1502 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:27.870951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd97f3c-abe5-4629-92d0-b951903d1502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:27.872419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd97f3c-abe5-4629-92d0-b951903d1502 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:27.874093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 28/60] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:27.879725 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf3389-17ee-4837-8d12-7a388e539270 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:27.980963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf3389-17ee-4837-8d12-7a388e539270 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:28.018150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398a997-6eeb-407e-aaee-f816ee9a9c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:28.018808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398a997-6eeb-407e-aaee-f816ee9a9c2b None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:28.019088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398a997-6eeb-407e-aaee-f816ee9a9c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:28.019391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398a997-6eeb-407e-aaee-f816ee9a9c2b None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:28.020092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398a997-6eeb-407e-aaee-f816ee9a9c2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:28.021545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398a997-6eeb-407e-aaee-f816ee9a9c2b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:28.045721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:28.052924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 33/61] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:40:27 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:31.348900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2392dd-d967-457c-8107-b6bb1a28ea56 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.349926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2392dd-d967-457c-8107-b6bb1a28ea56 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.350400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2392dd-d967-457c-8107-b6bb1a28ea56 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.352342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 29/62] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:31.357794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f550e2-4b58-472b-91e5-bd7be2ac0968 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.358770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f550e2-4b58-472b-91e5-bd7be2ac0968 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.359280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f550e2-4b58-472b-91e5-bd7be2ac0968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.374285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781421db-d475-4d93-8c02-b53cec57ec20 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.374888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781421db-d475-4d93-8c02-b53cec57ec20 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.375422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781421db-d475-4d93-8c02-b53cec57ec20 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.377255 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 30/63] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:31.382506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cffc2f-296a-41d4-8d62-eae8dc126b5b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.383049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cffc2f-296a-41d4-8d62-eae8dc126b5b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.383534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cffc2f-296a-41d4-8d62-eae8dc126b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.403504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-53f550e2-4b58-472b-91e5-bd7be2ac0968 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:31.405302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-53f550e2-4b58-472b-91e5-bd7be2ac0968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:31.412971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-b4cffc2f-296a-41d4-8d62-eae8dc126b5b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:31.414484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b4cffc2f-296a-41d4-8d62-eae8dc126b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:31.531104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 34/64] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:31.533772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350bf9-4d79-472d-86ea-c144ef4f7bc0 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.534917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350bf9-4d79-472d-86ea-c144ef4f7bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.535530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350bf9-4d79-472d-86ea-c144ef4f7bc0 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.538325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 35/65] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:31.545504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80312499-c74b-4161-8004-d305243c01e5 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.547657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80312499-c74b-4161-8004-d305243c01e5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.548975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80312499-c74b-4161-8004-d305243c01e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.555005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 31/66] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:31.559667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cffc2f-296a-41d4-8d62-eae8dc126b5b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:31.573959 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-80312499-c74b-4161-8004-d305243c01e5 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:31.575377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-80312499-c74b-4161-8004-d305243c01e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:31.670914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 36/67] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:31.672197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80312499-c74b-4161-8004-d305243c01e5 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:31.676954 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cffc2f-296a-41d4-8d62-eae8dc126b5b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:31.697937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:31.731046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397c007-0ce3-4b10-8a3b-a70790f2def1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:31.731910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397c007-0ce3-4b10-8a3b-a70790f2def1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.732199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397c007-0ce3-4b10-8a3b-a70790f2def1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.732482 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397c007-0ce3-4b10-8a3b-a70790f2def1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.743178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397c007-0ce3-4b10-8a3b-a70790f2def1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:31.745222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-2397c007-0ce3-4b10-8a3b-a70790f2def1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:31.751784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:31.752715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:31.753495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 32/68] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:40:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:31.754315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd65d0-31ba-4074-a316-4280280af426 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.754742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd65d0-31ba-4074-a316-4280280af426 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.755175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd65d0-31ba-4074-a316-4280280af426 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.756495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 33/69] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:31.760053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd65d0-31ba-4074-a316-4280280af426 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:31.799233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80312499-c74b-4161-8004-d305243c01e5 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:31.835275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d68e25-9710-4b94-8d51-1f403e58e65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:31.836102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68e25-9710-4b94-8d51-1f403e58e65e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.836544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68e25-9710-4b94-8d51-1f403e58e65e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.836961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68e25-9710-4b94-8d51-1f403e58e65e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.848859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d68e25-9710-4b94-8d51-1f403e58e65e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-21d68e25-9710-4b94-8d51-1f403e58e65e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:31.857952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:31.858850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:31.859661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 37/70] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:40:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:31.860286 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d68e25-9710-4b94-8d51-1f403e58e65e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:31.872448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd65d0-31ba-4074-a316-4280280af426 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:31.906456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8741a-4cfe-460b-9946-b3b29a4239f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:31.907293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8741a-4cfe-460b-9946-b3b29a4239f9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.907590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8741a-4cfe-460b-9946-b3b29a4239f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.907868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8741a-4cfe-460b-9946-b3b29a4239f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.918220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8741a-4cfe-460b-9946-b3b29a4239f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-b8a8741a-4cfe-460b-9946-b3b29a4239f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:31.926104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:31.927045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:31.927846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 34/71] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:31.928487 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342464df-d969-4457-90ea-a63f306059ce None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:31.928770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342464df-d969-4457-90ea-a63f306059ce None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:31.928993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342464df-d969-4457-90ea-a63f306059ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:31.945229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-342464df-d969-4457-90ea-a63f306059ce None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:31.946460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-342464df-d969-4457-90ea-a63f306059ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:31.968557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d68e25-9710-4b94-8d51-1f403e58e65e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:31.969003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:31.988389 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:32.002928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9139d3-3b07-45f2-a1cf-de93129c1081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.003642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9139d3-3b07-45f2-a1cf-de93129c1081 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.003929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9139d3-3b07-45f2-a1cf-de93129c1081 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.004262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9139d3-3b07-45f2-a1cf-de93129c1081 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.005020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9139d3-3b07-45f2-a1cf-de93129c1081 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:32.005381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9139d3-3b07-45f2-a1cf-de93129c1081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:32.006806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9139d3-3b07-45f2-a1cf-de93129c1081 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:32.012354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 35/72] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:32.013366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342464df-d969-4457-90ea-a63f306059ce None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.023380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 38/73] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:40:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:32.023380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5806662-c21a-4b0f-a150-ca0a79477670 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.023380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5806662-c21a-4b0f-a150-ca0a79477670 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.023380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5806662-c21a-4b0f-a150-ca0a79477670 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.023380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 39/74] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:32.023380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5806662-c21a-4b0f-a150-ca0a79477670 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.111687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342464df-d969-4457-90ea-a63f306059ce None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:32.128059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5806662-c21a-4b0f-a150-ca0a79477670 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:32.144824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb400490-1a32-4a5b-8bfe-b773cce37950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.145502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb400490-1a32-4a5b-8bfe-b773cce37950 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.145790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb400490-1a32-4a5b-8bfe-b773cce37950 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.146070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb400490-1a32-4a5b-8bfe-b773cce37950 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.146819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb400490-1a32-4a5b-8bfe-b773cce37950 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:32.147131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb400490-1a32-4a5b-8bfe-b773cce37950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:32.148551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb400490-1a32-4a5b-8bfe-b773cce37950 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:32.156703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 36/75] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:32.156703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb400490-1a32-4a5b-8bfe-b773cce37950 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.168403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ae255-a3f0-467e-aa1a-d296172a7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.169057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ae255-a3f0-467e-aa1a-d296172a7c8f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.169345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ae255-a3f0-467e-aa1a-d296172a7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.169623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ae255-a3f0-467e-aa1a-d296172a7c8f None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.170410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ae255-a3f0-467e-aa1a-d296172a7c8f None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:32.170711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ae255-a3f0-467e-aa1a-d296172a7c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:32.172146 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ae255-a3f0-467e-aa1a-d296172a7c8f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:32.178915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:32.180460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 40/76] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:32.181816 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af25c-63c3-4e13-8ecb-30ae0c65ce2a None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.182134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af25c-63c3-4e13-8ecb-30ae0c65ce2a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.182400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af25c-63c3-4e13-8ecb-30ae0c65ce2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.198543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-f37af25c-63c3-4e13-8ecb-30ae0c65ce2a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:32.199782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f37af25c-63c3-4e13-8ecb-30ae0c65ce2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:32.238007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:32.262356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb400490-1a32-4a5b-8bfe-b773cce37950 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:32.266617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 41/77] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:32.268700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af25c-63c3-4e13-8ecb-30ae0c65ce2a None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.297988 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed876cc5-5dfb-41ae-ac12-fba94d71fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.298880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed876cc5-5dfb-41ae-ac12-fba94d71fe1d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.299164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed876cc5-5dfb-41ae-ac12-fba94d71fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.299472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed876cc5-5dfb-41ae-ac12-fba94d71fe1d None admin] PATH_INFO: `/v3/users/b052f32177f94f9ea753f4978551e6df` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.314305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed876cc5-5dfb-41ae-ac12-fba94d71fe1d None admin] RBAC: Authorizing `identity:get_user(user_id=b052f32177f94f9ea753f4978551e6df)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:32.315834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed876cc5-5dfb-41ae-ac12-fba94d71fe1d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:32.317540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 37/78] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/users/b052f32177f94f9ea753f4978551e6df => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:32.318311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed876cc5-5dfb-41ae-ac12-fba94d71fe1d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.371654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af25c-63c3-4e13-8ecb-30ae0c65ce2a None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:32.406343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563f2fc-b49d-422f-9fb2-d354451d5bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.406530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:32.407182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563f2fc-b49d-422f-9fb2-d354451d5bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.407544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563f2fc-b49d-422f-9fb2-d354451d5bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.407865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563f2fc-b49d-422f-9fb2-d354451d5bb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.418469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563f2fc-b49d-422f-9fb2-d354451d5bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-b563f2fc-b49d-422f-9fb2-d354451d5bb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:32.426200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:32.427055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:32.427927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 42/79] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:32.428541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563f2fc-b49d-422f-9fb2-d354451d5bb3 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.435045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed876cc5-5dfb-41ae-ac12-fba94d71fe1d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:32.468572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:32.470590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba738791-897b-4c49-830f-aeb430ece1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.471601 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba738791-897b-4c49-830f-aeb430ece1fc None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.471864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba738791-897b-4c49-830f-aeb430ece1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.472159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba738791-897b-4c49-830f-aeb430ece1fc None admin] PATH_INFO: `/v3/users/825e62d400e54cc0868912432c257354` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.473440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba738791-897b-4c49-830f-aeb430ece1fc None admin] RBAC: Authorizing `identity:get_user(user_id=825e62d400e54cc0868912432c257354)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:32.474921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba738791-897b-4c49-830f-aeb430ece1fc None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:32.476587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 38/80] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/users/825e62d400e54cc0868912432c257354 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:32.477412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba738791-897b-4c49-830f-aeb430ece1fc None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.542077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563f2fc-b49d-422f-9fb2-d354451d5bb3 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:32.575094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df80c29-07c5-46db-9ded-16422409d159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.576011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df80c29-07c5-46db-9ded-16422409d159 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.576296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df80c29-07c5-46db-9ded-16422409d159 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.576572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df80c29-07c5-46db-9ded-16422409d159 None admin] PATH_INFO: `/v3/users/b052f32177f94f9ea753f4978551e6df` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.576886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba738791-897b-4c49-830f-aeb430ece1fc None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:32.577754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df80c29-07c5-46db-9ded-16422409d159 None admin] RBAC: Authorizing `identity:get_user(user_id=b052f32177f94f9ea753f4978551e6df)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:32.579232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df80c29-07c5-46db-9ded-16422409d159 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:32.580905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 43/81] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/users/b052f32177f94f9ea753f4978551e6df => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:32.581673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df80c29-07c5-46db-9ded-16422409d159 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.612146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed01d8b-a90a-449c-ac62-3803a9b9e737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.612953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed01d8b-a90a-449c-ac62-3803a9b9e737 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.613238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed01d8b-a90a-449c-ac62-3803a9b9e737 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.613515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed01d8b-a90a-449c-ac62-3803a9b9e737 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.614696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed01d8b-a90a-449c-ac62-3803a9b9e737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:32.616197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed01d8b-a90a-449c-ac62-3803a9b9e737 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:32.617834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 39/82] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:32.618656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed01d8b-a90a-449c-ac62-3803a9b9e737 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.633482 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:32.682517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df80c29-07c5-46db-9ded-16422409d159 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:32.684104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:32.716840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed01d8b-a90a-449c-ac62-3803a9b9e737 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:32.717356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2ee05-eb5e-450c-a9d5-1006a4b123bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.718007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2ee05-eb5e-450c-a9d5-1006a4b123bb None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.718291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2ee05-eb5e-450c-a9d5-1006a4b123bb None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.718564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2ee05-eb5e-450c-a9d5-1006a4b123bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.750015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a34ea-cdb4-4614-97b4-63d44881053a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.750693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a34ea-cdb4-4614-97b4-63d44881053a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.750982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a34ea-cdb4-4614-97b4-63d44881053a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.751276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a34ea-cdb4-4614-97b4-63d44881053a None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.752028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a34ea-cdb4-4614-97b4-63d44881053a None admin] RBAC: Adding query filter params (name=member) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:32.752339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a34ea-cdb4-4614-97b4-63d44881053a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:32.753790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a34ea-cdb4-4614-97b4-63d44881053a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:32.766348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 40/83] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:32.766348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a34ea-cdb4-4614-97b4-63d44881053a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.787691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2ee05-eb5e-450c-a9d5-1006a4b123bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:32.789221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2ee05-eb5e-450c-a9d5-1006a4b123bb None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:32.844992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:32.868923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a34ea-cdb4-4614-97b4-63d44881053a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:32.887894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 44/84] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:32.888750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2ee05-eb5e-450c-a9d5-1006a4b123bb None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:32.890941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:32.903048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600803ac-9361-433a-8f60-7d0e8be39299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:32.903746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600803ac-9361-433a-8f60-7d0e8be39299 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:32.904035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600803ac-9361-433a-8f60-7d0e8be39299 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:32.904316 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600803ac-9361-433a-8f60-7d0e8be39299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:32.970033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600803ac-9361-433a-8f60-7d0e8be39299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:32.971484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600803ac-9361-433a-8f60-7d0e8be39299 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:32.989526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2ee05-eb5e-450c-a9d5-1006a4b123bb None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:33.022083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f909556-1161-4af8-87b9-da9d9d85236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:33.022743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f909556-1161-4af8-87b9-da9d9d85236f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:33.023029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f909556-1161-4af8-87b9-da9d9d85236f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:33.023333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f909556-1161-4af8-87b9-da9d9d85236f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:33.042890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:33.065782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 41/85] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:33.066592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600803ac-9361-433a-8f60-7d0e8be39299 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:33.092485 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f909556-1161-4af8-87b9-da9d9d85236f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:33.094189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f909556-1161-4af8-87b9-da9d9d85236f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:33.096088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:33.174763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600803ac-9361-433a-8f60-7d0e8be39299 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:33.200566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 45/86] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:33.201419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f909556-1161-4af8-87b9-da9d9d85236f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:33.212810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd897df-31a4-49d1-ab1c-a3088c2695f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:33.213493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd897df-31a4-49d1-ab1c-a3088c2695f9 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:33.213781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd897df-31a4-49d1-ab1c-a3088c2695f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:33.214066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd897df-31a4-49d1-ab1c-a3088c2695f9 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:33.214800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd897df-31a4-49d1-ab1c-a3088c2695f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:33.216297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd897df-31a4-49d1-ab1c-a3088c2695f9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:33.242441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 42/87] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:40:33 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:33.243321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd897df-31a4-49d1-ab1c-a3088c2695f9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:33.277901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:33.312076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f909556-1161-4af8-87b9-da9d9d85236f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:33.313976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:33.353452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe678d-313d-42a9-ba80-6873e27de719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:33.354342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe678d-313d-42a9-ba80-6873e27de719 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:33.354620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe678d-313d-42a9-ba80-6873e27de719 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:33.354891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe678d-313d-42a9-ba80-6873e27de719 None admin] PATH_INFO: `/v3/users/b052f32177f94f9ea753f4978551e6df` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:33.356159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe678d-313d-42a9-ba80-6873e27de719 None admin] RBAC: Authorizing `identity:get_user(user_id=b052f32177f94f9ea753f4978551e6df)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:33.357672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe678d-313d-42a9-ba80-6873e27de719 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:33.364309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 46/88] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:33 2022] GET /identity/v3/users/b052f32177f94f9ea753f4978551e6df => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:33.364489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd897df-31a4-49d1-ab1c-a3088c2695f9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:33.364565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe678d-313d-42a9-ba80-6873e27de719 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:33.404757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ded380-9802-494f-ac30-18c3eba958e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:33.405664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ded380-9802-494f-ac30-18c3eba958e6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:33.405938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ded380-9802-494f-ac30-18c3eba958e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:33.406217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ded380-9802-494f-ac30-18c3eba958e6 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:33.407557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ded380-9802-494f-ac30-18c3eba958e6 None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:33.409030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ded380-9802-494f-ac30-18c3eba958e6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:33.410763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 43/89] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:33 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:33.411648 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ded380-9802-494f-ac30-18c3eba958e6 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:33.506798 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe678d-313d-42a9-ba80-6873e27de719 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:33.541142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:33.546264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ded380-9802-494f-ac30-18c3eba958e6 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:33.555931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914c654-fc4f-461a-9b67-dc30126d9a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:33.557244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914c654-fc4f-461a-9b67-dc30126d9a4b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:33.557899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914c654-fc4f-461a-9b67-dc30126d9a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:33.558182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914c654-fc4f-461a-9b67-dc30126d9a4b None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:33.560269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914c654-fc4f-461a-9b67-dc30126d9a4b None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:33.561270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914c654-fc4f-461a-9b67-dc30126d9a4b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:33.563314 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 47/90] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:33 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:33.564050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914c654-fc4f-461a-9b67-dc30126d9a4b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:33.587036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774c104-5339-4a5c-96f0-a2f7510f012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:33.588144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c104-5339-4a5c-96f0-a2f7510f012b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:33.588425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c104-5339-4a5c-96f0-a2f7510f012b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:33.588704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c104-5339-4a5c-96f0-a2f7510f012b None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:33.589901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774c104-5339-4a5c-96f0-a2f7510f012b None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:33.591401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774c104-5339-4a5c-96f0-a2f7510f012b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:33.593125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 44/91] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:33 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:33.593894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774c104-5339-4a5c-96f0-a2f7510f012b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:33.617035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:33.694134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914c654-fc4f-461a-9b67-dc30126d9a4b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:33.729515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774c104-5339-4a5c-96f0-a2f7510f012b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:33.745046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4203e7-4b1a-40d0-af70-bc7c2793736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:33.746330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4203e7-4b1a-40d0-af70-bc7c2793736c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:33.746808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4203e7-4b1a-40d0-af70-bc7c2793736c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:33.747502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4203e7-4b1a-40d0-af70-bc7c2793736c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:33.771777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b603ff0-504c-413a-8691-ddef35b92756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:33.773184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b603ff0-504c-413a-8691-ddef35b92756 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:33.773848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b603ff0-504c-413a-8691-ddef35b92756 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:33.774260 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b603ff0-504c-413a-8691-ddef35b92756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:33.775748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b603ff0-504c-413a-8691-ddef35b92756 None admin] RBAC: Adding query filter params (role.id=d2054df5a6d24f5d90793b9b81cd4b4b, scope.project.id=f35ff6000de749fc927ed896def15c04, user.id=b052f32177f94f9ea753f4978551e6df) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:33.776617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b603ff0-504c-413a-8691-ddef35b92756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:33.778724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b603ff0-504c-413a-8691-ddef35b92756 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:33.800175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 45/92] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:33 2022] GET /identity/v3/role_assignments?user.id=b052f32177f94f9ea753f4978551e6df&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=d2054df5a6d24f5d90793b9b81cd4b4b => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:33.800175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b603ff0-504c-413a-8691-ddef35b92756 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:33.844698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4203e7-4b1a-40d0-af70-bc7c2793736c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:33.846194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4203e7-4b1a-40d0-af70-bc7c2793736c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:33.891353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:33.896896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:33.969511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 48/93] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:33.970427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4203e7-4b1a-40d0-af70-bc7c2793736c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:34.039962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b603ff0-504c-413a-8691-ddef35b92756 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:34.101086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1ed39-e2a0-487c-a24f-f6dd219a60f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:34.105788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1ed39-e2a0-487c-a24f-f6dd219a60f1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:34.105898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1ed39-e2a0-487c-a24f-f6dd219a60f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:34.106342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1ed39-e2a0-487c-a24f-f6dd219a60f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:34.107687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1ed39-e2a0-487c-a24f-f6dd219a60f1 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=f35ff6000de749fc927ed896def15c04, user.id=825e62d400e54cc0868912432c257354) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:34.108006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1ed39-e2a0-487c-a24f-f6dd219a60f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:34.109477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1ed39-e2a0-487c-a24f-f6dd219a60f1 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:34.121663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 46/94] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:33 2022] GET /identity/v3/role_assignments?user.id=825e62d400e54cc0868912432c257354&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:34.122520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1ed39-e2a0-487c-a24f-f6dd219a60f1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:34.131107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4203e7-4b1a-40d0-af70-bc7c2793736c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:34.203183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66322818-c547-4392-9c57-5b5d12ed7ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:34.204274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66322818-c547-4392-9c57-5b5d12ed7ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:34.204748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66322818-c547-4392-9c57-5b5d12ed7ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:34.205208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66322818-c547-4392-9c57-5b5d12ed7ea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:34.206277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66322818-c547-4392-9c57-5b5d12ed7ea6 None admin] RBAC: Adding query filter params (role.id=d0a32856510a480a897677657ace20de, scope.project.id=f35ff6000de749fc927ed896def15c04, user.id=b052f32177f94f9ea753f4978551e6df) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:34.206806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66322818-c547-4392-9c57-5b5d12ed7ea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:34.208540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66322818-c547-4392-9c57-5b5d12ed7ea6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:34.221447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 49/95] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:33 2022] GET /identity/v3/role_assignments?user.id=b052f32177f94f9ea753f4978551e6df&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=d0a32856510a480a897677657ace20de => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:34.263202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:34.269351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1ed39-e2a0-487c-a24f-f6dd219a60f1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:34.315347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b3095-5743-400d-b0ae-6e8076e26274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:34.316172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b3095-5743-400d-b0ae-6e8076e26274 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:34.316463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b3095-5743-400d-b0ae-6e8076e26274 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:34.316740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b3095-5743-400d-b0ae-6e8076e26274 None admin] PATH_INFO: `/v3/projects/e1c18199281c403490a472221374dbac` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:34.317961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b3095-5743-400d-b0ae-6e8076e26274 None admin] RBAC: Authorizing `identity:get_project(project_id=e1c18199281c403490a472221374dbac)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:34.319860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b3095-5743-400d-b0ae-6e8076e26274 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:34.321760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 47/96] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:34 2022] GET /identity/v3/projects/e1c18199281c403490a472221374dbac => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:34.328429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66322818-c547-4392-9c57-5b5d12ed7ea6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:34.335059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:34.474557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66322818-c547-4392-9c57-5b5d12ed7ea6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:34.511169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcad69-2743-4c38-829f-03fdbb414f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:34.513060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcad69-2743-4c38-829f-03fdbb414f23 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:34.513554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcad69-2743-4c38-829f-03fdbb414f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:34.514039 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcad69-2743-4c38-829f-03fdbb414f23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:34.515070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fcad69-2743-4c38-829f-03fdbb414f23 None admin] RBAC: Adding query filter params (role.id=d0a32856510a480a897677657ace20de, scope.project.id=e1c18199281c403490a472221374dbac, user.id=b052f32177f94f9ea753f4978551e6df) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:34.515586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fcad69-2743-4c38-829f-03fdbb414f23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:34.517454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fcad69-2743-4c38-829f-03fdbb414f23 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:34.523875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 50/97] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:34 2022] GET /identity/v3/role_assignments?user.id=b052f32177f94f9ea753f4978551e6df&scope.project.id=e1c18199281c403490a472221374dbac&role.id=d0a32856510a480a897677657ace20de => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:35.962488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b744e99-653c-4dcc-8a66-2427956ec96f None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:35.962977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b744e99-653c-4dcc-8a66-2427956ec96f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:35.963049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b744e99-653c-4dcc-8a66-2427956ec96f None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:35.964732 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 48/98] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:35.980982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab811fe5-e18e-4526-9eab-91da2b5d6e71 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:35.981589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab811fe5-e18e-4526-9eab-91da2b5d6e71 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:35.981589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab811fe5-e18e-4526-9eab-91da2b5d6e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:35.999029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ab811fe5-e18e-4526-9eab-91da2b5d6e71 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:36.000293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ab811fe5-e18e-4526-9eab-91da2b5d6e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:36.004720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d13e1f-1150-4886-a938-81305e522a04 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.005007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d13e1f-1150-4886-a938-81305e522a04 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.005286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d13e1f-1150-4886-a938-81305e522a04 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:36.006853 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 49/99] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:40:36.026886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86b246-b5dd-4192-ac9b-e8200617b9bb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.027176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86b246-b5dd-4192-ac9b-e8200617b9bb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.027486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86b246-b5dd-4192-ac9b-e8200617b9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:36.050731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-7f86b246-b5dd-4192-ac9b-e8200617b9bb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:36.051967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7f86b246-b5dd-4192-ac9b-e8200617b9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:36.117482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:36.135156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 51/100] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:36.167966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 50/101] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:36.171426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab811fe5-e18e-4526-9eab-91da2b5d6e71 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:36.203608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86b246-b5dd-4192-ac9b-e8200617b9bb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:36.360105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:36.367386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86b246-b5dd-4192-ac9b-e8200617b9bb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:36.397440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab811fe5-e18e-4526-9eab-91da2b5d6e71 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:36.409018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd76d62-beac-49ca-988e-f5a11b016ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:36.409841 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd76d62-beac-49ca-988e-f5a11b016ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.410126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd76d62-beac-49ca-988e-f5a11b016ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.410482 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd76d62-beac-49ca-988e-f5a11b016ea6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:36.432406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd76d62-beac-49ca-988e-f5a11b016ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-1dd76d62-beac-49ca-988e-f5a11b016ea6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:36.448389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:36.449271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:36.450103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 51/102] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:40:36.455292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd76d62-beac-49ca-988e-f5a11b016ea6 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:36.483984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e48407-ad37-40b4-843a-a71640845840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:36.484827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e48407-ad37-40b4-843a-a71640845840 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.485121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e48407-ad37-40b4-843a-a71640845840 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.485403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e48407-ad37-40b4-843a-a71640845840 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:36.486920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e48407-ad37-40b4-843a-a71640845840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:36.488451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e48407-ad37-40b4-843a-a71640845840 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:36.490113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 52/103] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:36.495791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e48407-ad37-40b4-843a-a71640845840 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:36.566391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd76d62-beac-49ca-988e-f5a11b016ea6 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:36.608087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dae58-d6e1-4fbf-a069-eb3ce3c05aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:36.608776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dae58-d6e1-4fbf-a069-eb3ce3c05aac None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.609063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dae58-d6e1-4fbf-a069-eb3ce3c05aac None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.609337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dae58-d6e1-4fbf-a069-eb3ce3c05aac None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:36.610100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dae58-d6e1-4fbf-a069-eb3ce3c05aac None admin] RBAC: Adding query filter params (name=reader) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:36.610467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dae58-d6e1-4fbf-a069-eb3ce3c05aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:36.611966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dae58-d6e1-4fbf-a069-eb3ce3c05aac None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:36.620926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 52/104] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:36.626367 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dae58-d6e1-4fbf-a069-eb3ce3c05aac None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:36.652248 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:36.654672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:36.722336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e48407-ad37-40b4-843a-a71640845840 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:36.786814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b369de-4d22-4ab6-941d-bfebc1e1d34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:36.787536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b369de-4d22-4ab6-941d-bfebc1e1d34a None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.787830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b369de-4d22-4ab6-941d-bfebc1e1d34a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.788114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b369de-4d22-4ab6-941d-bfebc1e1d34a None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:36.788855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b369de-4d22-4ab6-941d-bfebc1e1d34a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:36.790456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b369de-4d22-4ab6-941d-bfebc1e1d34a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:36.808593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dae58-d6e1-4fbf-a069-eb3ce3c05aac None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:36.839668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 53/105] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:40:36 2022] POST /identity/v3/projects => generated 302 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:40:36.842370 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6406e-2e21-4c9c-98a3-b8f2cdc5c425 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.842682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6406e-2e21-4c9c-98a3-b8f2cdc5c425 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.843019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6406e-2e21-4c9c-98a3-b8f2cdc5c425 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:36.846589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 54/106] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:36.852813 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51cf7d-366f-4abd-b6f9-34c727d11f2b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.853148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51cf7d-366f-4abd-b6f9-34c727d11f2b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.853412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51cf7d-366f-4abd-b6f9-34c727d11f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:36.877507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6e51cf7d-366f-4abd-b6f9-34c727d11f2b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:36.878649 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6e51cf7d-366f-4abd-b6f9-34c727d11f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:36.886720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc932d9f-dc66-470a-a68a-11fc43ee1be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:36.887661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc932d9f-dc66-470a-a68a-11fc43ee1be2 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.887951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc932d9f-dc66-470a-a68a-11fc43ee1be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.888231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc932d9f-dc66-470a-a68a-11fc43ee1be2 None admin] PATH_INFO: `/v3/users/8e6df2e0e74b412988d96dc876c164de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:36.916958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc932d9f-dc66-470a-a68a-11fc43ee1be2 None admin] RBAC: Authorizing `identity:get_user(user_id=8e6df2e0e74b412988d96dc876c164de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:36.918463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc932d9f-dc66-470a-a68a-11fc43ee1be2 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:36.920461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 53/107] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:36 2022] GET /identity/v3/users/8e6df2e0e74b412988d96dc876c164de => generated 289 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:36.926214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc932d9f-dc66-470a-a68a-11fc43ee1be2 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:36.952960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 55/108] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:36.962687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c9ee2-49b0-42ee-a272-68661320de59 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.962998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c9ee2-49b0-42ee-a272-68661320de59 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.963280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c9ee2-49b0-42ee-a272-68661320de59 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:36.964871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 56/109] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:36.976071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a2772-375f-4d41-a7d8-d28e5bb19f68 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:36.976396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a2772-375f-4d41-a7d8-d28e5bb19f68 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:36.976645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a2772-375f-4d41-a7d8-d28e5bb19f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.002581 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-765a2772-375f-4d41-a7d8-d28e5bb19f68 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:37.003866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-765a2772-375f-4d41-a7d8-d28e5bb19f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:37.059701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:37.078039 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 57/110] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:37.083884 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a2772-375f-4d41-a7d8-d28e5bb19f68 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:37.123810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:37.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc932d9f-dc66-470a-a68a-11fc43ee1be2 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:37.189053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3713eb-1392-41ec-a167-8953a41dcec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:37.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3713eb-1392-41ec-a167-8953a41dcec5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:37.190395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3713eb-1392-41ec-a167-8953a41dcec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:37.190687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3713eb-1392-41ec-a167-8953a41dcec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.218583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a2772-375f-4d41-a7d8-d28e5bb19f68 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:37.263996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ebdec-dfb3-4b2c-9b2e-d786a837af60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:37.266396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ebdec-dfb3-4b2c-9b2e-d786a837af60 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:37.266396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ebdec-dfb3-4b2c-9b2e-d786a837af60 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:37.266396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ebdec-dfb3-4b2c-9b2e-d786a837af60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.277450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ebdec-dfb3-4b2c-9b2e-d786a837af60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-9b1ebdec-dfb3-4b2c-9b2e-d786a837af60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:37.286265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:37.287380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 58/111] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:40:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:37.288406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ebdec-dfb3-4b2c-9b2e-d786a837af60 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:37.314854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3713eb-1392-41ec-a167-8953a41dcec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:37.316699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3713eb-1392-41ec-a167-8953a41dcec5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:37.329363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:37.405437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ebdec-dfb3-4b2c-9b2e-d786a837af60 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:37.451761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:37.458216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c6c7e-bb5f-488e-b389-5049ae109356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:37.460090 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c6c7e-bb5f-488e-b389-5049ae109356 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:37.460368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c6c7e-bb5f-488e-b389-5049ae109356 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:37.460740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c6c7e-bb5f-488e-b389-5049ae109356 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.472855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c6c7e-bb5f-488e-b389-5049ae109356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:37.480232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 54/112] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 05:40:37.481248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3713eb-1392-41ec-a167-8953a41dcec5 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-b03c6c7e-bb5f-488e-b389-5049ae109356 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:37.486203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:37.487089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:37.487877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:37.488450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 59/113] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:37.489501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ef94-fba0-4081-ac8c-146ab83313b6 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:37.489792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ef94-fba0-4081-ac8c-146ab83313b6 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:37.490072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ef94-fba0-4081-ac8c-146ab83313b6 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.495620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 60/114] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:37.496457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44ef94-fba0-4081-ac8c-146ab83313b6 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:37.587751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3713eb-1392-41ec-a167-8953a41dcec5 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:37.607266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:37.615575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44ef94-fba0-4081-ac8c-146ab83313b6 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:37.623315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4161e6-1480-458d-8467-82e6c0178e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:37.624302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4161e6-1480-458d-8467-82e6c0178e84 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:37.624779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4161e6-1480-458d-8467-82e6c0178e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:37.625246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4161e6-1480-458d-8467-82e6c0178e84 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.626238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4161e6-1480-458d-8467-82e6c0178e84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:37.626740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4161e6-1480-458d-8467-82e6c0178e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:37.628369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4161e6-1480-458d-8467-82e6c0178e84 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:37.637432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 55/115] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:40:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:37.638489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4161e6-1480-458d-8467-82e6c0178e84 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:37.657170 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a256966-0005-49da-b7cb-8844ae8113b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:37.658109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a256966-0005-49da-b7cb-8844ae8113b6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:37.658436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a256966-0005-49da-b7cb-8844ae8113b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:37.658716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a256966-0005-49da-b7cb-8844ae8113b6 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.660166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a256966-0005-49da-b7cb-8844ae8113b6 None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:37.661713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a256966-0005-49da-b7cb-8844ae8113b6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:37.668706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 61/116] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:37 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:37.670228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b48bf8-e5c9-4ad7-bc78-6329ed262632 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:37.670441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b48bf8-e5c9-4ad7-bc78-6329ed262632 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:37.670731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b48bf8-e5c9-4ad7-bc78-6329ed262632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.677912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:37.691063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-46b48bf8-e5c9-4ad7-bc78-6329ed262632 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:37.692656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-46b48bf8-e5c9-4ad7-bc78-6329ed262632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:37.746172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4161e6-1480-458d-8467-82e6c0178e84 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:37.764440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 62/117] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:37.772311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b48bf8-e5c9-4ad7-bc78-6329ed262632 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:37.787267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98e75a-8502-4abb-a013-034e6fcb6944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:37.787267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98e75a-8502-4abb-a013-034e6fcb6944 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:37.787267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98e75a-8502-4abb-a013-034e6fcb6944 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:37.787267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98e75a-8502-4abb-a013-034e6fcb6944 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.787669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98e75a-8502-4abb-a013-034e6fcb6944 None admin] RBAC: Adding query filter params (name=member) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:37.787669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98e75a-8502-4abb-a013-034e6fcb6944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:37.787669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98e75a-8502-4abb-a013-034e6fcb6944 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:37.793189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 56/118] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:37.794111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98e75a-8502-4abb-a013-034e6fcb6944 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:37.884596 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:37.892900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b48bf8-e5c9-4ad7-bc78-6329ed262632 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:37.898751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:37.903124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98e75a-8502-4abb-a013-034e6fcb6944 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:37.932973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f94f5-4719-49e9-92e0-52c3bf0c22e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:37.933789 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f94f5-4719-49e9-92e0-52c3bf0c22e4 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:37.934075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f94f5-4719-49e9-92e0-52c3bf0c22e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:37.934352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f94f5-4719-49e9-92e0-52c3bf0c22e4 None admin] PATH_INFO: `/v3/users/b052f32177f94f9ea753f4978551e6df` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.935700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f94f5-4719-49e9-92e0-52c3bf0c22e4 None admin] RBAC: Authorizing `identity:get_user(user_id=b052f32177f94f9ea753f4978551e6df)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:37.937190 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f94f5-4719-49e9-92e0-52c3bf0c22e4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:37.943299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 63/119] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:37 2022] GET /identity/v3/users/b052f32177f94f9ea753f4978551e6df => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:37.943299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f94f5-4719-49e9-92e0-52c3bf0c22e4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:37.944595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2e2f6-9ce9-481d-9a4f-bba2f600900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:37.945268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2e2f6-9ce9-481d-9a4f-bba2f600900b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:37.945552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2e2f6-9ce9-481d-9a4f-bba2f600900b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:37.945829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2e2f6-9ce9-481d-9a4f-bba2f600900b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:37.946704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2e2f6-9ce9-481d-9a4f-bba2f600900b None admin] RBAC: Adding query filter params (role.id=8f325e9c371744daace594fefd8d1279, scope.project.id=f35ff6000de749fc927ed896def15c04, user.id=8e6df2e0e74b412988d96dc876c164de) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:37.947016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2e2f6-9ce9-481d-9a4f-bba2f600900b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:37.948521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2e2f6-9ce9-481d-9a4f-bba2f600900b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:37.954433 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 57/120] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:37 2022] GET /identity/v3/role_assignments?user.id=8e6df2e0e74b412988d96dc876c164de&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=8f325e9c371744daace594fefd8d1279 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:37.963396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2e2f6-9ce9-481d-9a4f-bba2f600900b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:38.089852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f94f5-4719-49e9-92e0-52c3bf0c22e4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:38.090683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2e2f6-9ce9-481d-9a4f-bba2f600900b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:38.139115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ba90b-4d55-49b8-95af-df723bcc70f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:38.140201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ba90b-4d55-49b8-95af-df723bcc70f8 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:38.140561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ba90b-4d55-49b8-95af-df723bcc70f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:38.140909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ba90b-4d55-49b8-95af-df723bcc70f8 None admin] PATH_INFO: `/v3/users/b052f32177f94f9ea753f4978551e6df` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:38.142221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ba90b-4d55-49b8-95af-df723bcc70f8 None admin] RBAC: Authorizing `identity:get_user(user_id=b052f32177f94f9ea753f4978551e6df)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:38.143814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ba90b-4d55-49b8-95af-df723bcc70f8 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:38.145551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 64/121] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:37 2022] GET /identity/v3/users/b052f32177f94f9ea753f4978551e6df => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:38.146828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6b22-b409-4ffc-9ca7-d03b0a2f0e64 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:38.147209 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6b22-b409-4ffc-9ca7-d03b0a2f0e64 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:38.147561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6b22-b409-4ffc-9ca7-d03b0a2f0e64 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:38.148877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 65/122] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:38.149311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e4bfe-b05a-4cbc-83d6-718d7701756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:38.149789 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c6b22-b409-4ffc-9ca7-d03b0a2f0e64 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:38.151098 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e4bfe-b05a-4cbc-83d6-718d7701756d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:38.151595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e4bfe-b05a-4cbc-83d6-718d7701756d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:38.152035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e4bfe-b05a-4cbc-83d6-718d7701756d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:38.167509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e4bfe-b05a-4cbc-83d6-718d7701756d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-500e4bfe-b05a-4cbc-83d6-718d7701756d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:38.178345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:38.179215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:38.180555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:38.181203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 58/123] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:40:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:38.186546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e4bfe-b05a-4cbc-83d6-718d7701756d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:38.190409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:38.217594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:38.299071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e4bfe-b05a-4cbc-83d6-718d7701756d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:38.302621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c6b22-b409-4ffc-9ca7-d03b0a2f0e64 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:38.335496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d771ce-7b5d-4665-8148-bb2c29b8031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:38.336379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d771ce-7b5d-4665-8148-bb2c29b8031f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:38.336657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d771ce-7b5d-4665-8148-bb2c29b8031f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:38.336990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d771ce-7b5d-4665-8148-bb2c29b8031f None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:38.338425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d771ce-7b5d-4665-8148-bb2c29b8031f None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:38.339974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d771ce-7b5d-4665-8148-bb2c29b8031f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:38.341880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 59/124] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:38 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:38.347608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7917f31-10cf-48ec-bd85-32f75b9fc316 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:38.347905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7917f31-10cf-48ec-bd85-32f75b9fc316 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:38.348176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7917f31-10cf-48ec-bd85-32f75b9fc316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:38.351197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f4ba2-c725-4fe7-a25e-d73815f9b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:38.352408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f4ba2-c725-4fe7-a25e-d73815f9b513 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:38.353215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f4ba2-c725-4fe7-a25e-d73815f9b513 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:38.353706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f4ba2-c725-4fe7-a25e-d73815f9b513 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:38.356086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f4ba2-c725-4fe7-a25e-d73815f9b513 None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:38.357824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f4ba2-c725-4fe7-a25e-d73815f9b513 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:38.360403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 66/125] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:38 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:38.361722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f4ba2-c725-4fe7-a25e-d73815f9b513 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:38.365400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d7917f31-10cf-48ec-bd85-32f75b9fc316 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:38.366630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d7917f31-10cf-48ec-bd85-32f75b9fc316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:38.444352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 60/126] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:38.445820 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7917f31-10cf-48ec-bd85-32f75b9fc316 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:38.480436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:38.485405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:38.490555 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f4ba2-c725-4fe7-a25e-d73815f9b513 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:38.536451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b673-ee46-4343-9282-d2c947d1eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:38.537227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b673-ee46-4343-9282-d2c947d1eb64 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:38.537581 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b673-ee46-4343-9282-d2c947d1eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:38.537924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b673-ee46-4343-9282-d2c947d1eb64 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:38.538755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b673-ee46-4343-9282-d2c947d1eb64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:38.539131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b673-ee46-4343-9282-d2c947d1eb64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:38.540707 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b673-ee46-4343-9282-d2c947d1eb64 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:38.549738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 67/127] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:38.550394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b673-ee46-4343-9282-d2c947d1eb64 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:38.590071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7917f31-10cf-48ec-bd85-32f75b9fc316 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:38.637032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e63cd-73ee-4adb-9c46-bcda131b3dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:38.637961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e63cd-73ee-4adb-9c46-bcda131b3dec None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:38.638240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e63cd-73ee-4adb-9c46-bcda131b3dec None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:38.638513 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e63cd-73ee-4adb-9c46-bcda131b3dec None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04/users/b052f32177f94f9ea753f4978551e6df/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:38.641360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e63cd-73ee-4adb-9c46-bcda131b3dec None admin] RBAC: Authorizing `identity:create_grant(project_id=f35ff6000de749fc927ed896def15c04, user_id=b052f32177f94f9ea753f4978551e6df, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:38.642840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e63cd-73ee-4adb-9c46-bcda131b3dec None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:38.661135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 61/128] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:40:38 2022] PUT /identity/v3/projects/f35ff6000de749fc927ed896def15c04/users/b052f32177f94f9ea753f4978551e6df/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:40:38.667546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e63cd-73ee-4adb-9c46-bcda131b3dec None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:38.670147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b673-ee46-4343-9282-d2c947d1eb64 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:38.741149 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1f540-7d7b-4bbd-aa5e-b8d022c070b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:38.742257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1f540-7d7b-4bbd-aa5e-b8d022c070b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:38.742577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1f540-7d7b-4bbd-aa5e-b8d022c070b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:38.742900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1f540-7d7b-4bbd-aa5e-b8d022c070b7 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04/users/b052f32177f94f9ea753f4978551e6df/roles/d2054df5a6d24f5d90793b9b81cd4b4b` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:38.750537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1f540-7d7b-4bbd-aa5e-b8d022c070b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f35ff6000de749fc927ed896def15c04, user_id=b052f32177f94f9ea753f4978551e6df, role_id=d2054df5a6d24f5d90793b9b81cd4b4b)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:38.752167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1f540-7d7b-4bbd-aa5e-b8d022c070b7 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:38.763944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:38.788675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e63cd-73ee-4adb-9c46-bcda131b3dec None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:38.792423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 68/129] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:40:38 2022] PUT /identity/v3/projects/f35ff6000de749fc927ed896def15c04/users/b052f32177f94f9ea753f4978551e6df/roles/d2054df5a6d24f5d90793b9b81cd4b4b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:40:38.793366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1f540-7d7b-4bbd-aa5e-b8d022c070b7 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:38.829210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125148b2-60bd-4176-8052-9801616f5455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:38.830053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125148b2-60bd-4176-8052-9801616f5455 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:38.830340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125148b2-60bd-4176-8052-9801616f5455 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:38.830617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125148b2-60bd-4176-8052-9801616f5455 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:38.839603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:38.843972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125148b2-60bd-4176-8052-9801616f5455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-125148b2-60bd-4176-8052-9801616f5455 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:38.852370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:38.853376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:38.854218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:38.859282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 62/130] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:38.861793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125148b2-60bd-4176-8052-9801616f5455 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:38.979500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125148b2-60bd-4176-8052-9801616f5455 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:39.000427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1f540-7d7b-4bbd-aa5e-b8d022c070b7 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:39.018827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce476746-ea58-4081-9209-ed373b3d239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:39.019659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce476746-ea58-4081-9209-ed373b3d239e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:39.019952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce476746-ea58-4081-9209-ed373b3d239e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:39.020234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce476746-ea58-4081-9209-ed373b3d239e None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:39.021014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce476746-ea58-4081-9209-ed373b3d239e None admin] RBAC: Adding query filter params (name=member) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:39.021327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce476746-ea58-4081-9209-ed373b3d239e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:39.022781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce476746-ea58-4081-9209-ed373b3d239e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:39.032592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 63/131] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:39.038509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce476746-ea58-4081-9209-ed373b3d239e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:39.041576 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:39.063763 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1698f-a851-4c43-b73f-cc8a8ca78839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:39.064768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1698f-a851-4c43-b73f-cc8a8ca78839 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:39.065042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1698f-a851-4c43-b73f-cc8a8ca78839 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:39.065315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1698f-a851-4c43-b73f-cc8a8ca78839 None admin] PATH_INFO: `/v3/users/825e62d400e54cc0868912432c257354` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:39.066544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1698f-a851-4c43-b73f-cc8a8ca78839 None admin] RBAC: Authorizing `identity:get_user(user_id=825e62d400e54cc0868912432c257354)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:39.068099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1698f-a851-4c43-b73f-cc8a8ca78839 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:39.070333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 69/132] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:38 2022] GET /identity/v3/users/825e62d400e54cc0868912432c257354 => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:39.076119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1698f-a851-4c43-b73f-cc8a8ca78839 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:39.156405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce476746-ea58-4081-9209-ed373b3d239e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:39.208291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:39.221642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11180a17-a7f6-4fd8-ad73-9e0472648329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:39.222555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180a17-a7f6-4fd8-ad73-9e0472648329 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:39.222839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180a17-a7f6-4fd8-ad73-9e0472648329 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:39.223117 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180a17-a7f6-4fd8-ad73-9e0472648329 None admin] PATH_INFO: `/v3/users/b052f32177f94f9ea753f4978551e6df` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:39.224489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11180a17-a7f6-4fd8-ad73-9e0472648329 None admin] RBAC: Authorizing `identity:get_user(user_id=b052f32177f94f9ea753f4978551e6df)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:39.225972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11180a17-a7f6-4fd8-ad73-9e0472648329 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:39.229887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 64/133] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:39 2022] GET /identity/v3/users/b052f32177f94f9ea753f4978551e6df => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:39.236034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11180a17-a7f6-4fd8-ad73-9e0472648329 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:39.238827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1698f-a851-4c43-b73f-cc8a8ca78839 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:39.302774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f834ec9-53af-4d16-b485-e33b7feae2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:39.303609 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f834ec9-53af-4d16-b485-e33b7feae2ad None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:39.303894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f834ec9-53af-4d16-b485-e33b7feae2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:39.304172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f834ec9-53af-4d16-b485-e33b7feae2ad None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:39.306231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f834ec9-53af-4d16-b485-e33b7feae2ad None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:39.307751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f834ec9-53af-4d16-b485-e33b7feae2ad None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:39.309504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 70/134] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:39 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:39.315177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f834ec9-53af-4d16-b485-e33b7feae2ad None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:39.366002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:39.421375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11180a17-a7f6-4fd8-ad73-9e0472648329 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:39.484924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fad7c9-b939-4130-ac9f-6fe32b71c06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:39.485814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fad7c9-b939-4130-ac9f-6fe32b71c06c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:39.486100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fad7c9-b939-4130-ac9f-6fe32b71c06c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:39.486386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fad7c9-b939-4130-ac9f-6fe32b71c06c None admin] PATH_INFO: `/v3/projects/e1c18199281c403490a472221374dbac` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:39.488216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fad7c9-b939-4130-ac9f-6fe32b71c06c None admin] RBAC: Authorizing `identity:get_project(project_id=e1c18199281c403490a472221374dbac)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:39.489681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fad7c9-b939-4130-ac9f-6fe32b71c06c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:39.491437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 65/135] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:39 2022] GET /identity/v3/projects/e1c18199281c403490a472221374dbac => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:39.497104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f834ec9-53af-4d16-b485-e33b7feae2ad None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:39.497690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fad7c9-b939-4130-ac9f-6fe32b71c06c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:39.601411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81759aab-4796-4a8a-9044-11ba4391ae85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:39.602665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81759aab-4796-4a8a-9044-11ba4391ae85 None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:39.603227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81759aab-4796-4a8a-9044-11ba4391ae85 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:39.603724 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81759aab-4796-4a8a-9044-11ba4391ae85 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04/users/825e62d400e54cc0868912432c257354/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:39.608548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81759aab-4796-4a8a-9044-11ba4391ae85 None admin] RBAC: Authorizing `identity:create_grant(project_id=f35ff6000de749fc927ed896def15c04, user_id=825e62d400e54cc0868912432c257354, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:39.610285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81759aab-4796-4a8a-9044-11ba4391ae85 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:39.618572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fad7c9-b939-4130-ac9f-6fe32b71c06c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:39.644835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 71/136] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:40:39 2022] PUT /identity/v3/projects/f35ff6000de749fc927ed896def15c04/users/825e62d400e54cc0868912432c257354/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:40:39.653613 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd028a90-3ed2-4f18-bd7a-b5ad844e8afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:39.654552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd028a90-3ed2-4f18-bd7a-b5ad844e8afc None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:39.654834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd028a90-3ed2-4f18-bd7a-b5ad844e8afc None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:39.655169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd028a90-3ed2-4f18-bd7a-b5ad844e8afc None admin] PATH_INFO: `/v3/projects/e1c18199281c403490a472221374dbac/users/b052f32177f94f9ea753f4978551e6df/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:39.657444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd028a90-3ed2-4f18-bd7a-b5ad844e8afc None admin] RBAC: Authorizing `identity:create_grant(project_id=e1c18199281c403490a472221374dbac, user_id=b052f32177f94f9ea753f4978551e6df, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:39.658998 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd028a90-3ed2-4f18-bd7a-b5ad844e8afc None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:39.671551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:39.677101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 66/137] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:40:39 2022] PUT /identity/v3/projects/e1c18199281c403490a472221374dbac/users/b052f32177f94f9ea753f4978551e6df/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:40:39.729509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216cc4c-58a4-48ae-9565-1aa13ff8b283 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:39.730148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216cc4c-58a4-48ae-9565-1aa13ff8b283 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:39.730608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216cc4c-58a4-48ae-9565-1aa13ff8b283 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:39.732322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 72/138] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:39.738446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce69cdc-e29b-48f5-a7d5-5bfc7fa69417 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:39.738742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce69cdc-e29b-48f5-a7d5-5bfc7fa69417 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:39.739026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce69cdc-e29b-48f5-a7d5-5bfc7fa69417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:39.764489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-3ce69cdc-e29b-48f5-a7d5-5bfc7fa69417 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:39.765902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3ce69cdc-e29b-48f5-a7d5-5bfc7fa69417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:39.865162 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 67/139] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:39.906620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216cc4c-58a4-48ae-9565-1aa13ff8b283 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:39.932249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:40.024257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216cc4c-58a4-48ae-9565-1aa13ff8b283 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:40.060976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c3870-e3b5-48ce-88d5-54dee6488411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:40.062147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c3870-e3b5-48ce-88d5-54dee6488411 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:40.062638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c3870-e3b5-48ce-88d5-54dee6488411 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:40.063166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c3870-e3b5-48ce-88d5-54dee6488411 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:40.068067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c3870-e3b5-48ce-88d5-54dee6488411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:40.070199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c3870-e3b5-48ce-88d5-54dee6488411 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:40.072221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 73/140] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:40.077833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce69cdc-e29b-48f5-a7d5-5bfc7fa69417 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:40.222809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce69cdc-e29b-48f5-a7d5-5bfc7fa69417 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:40.255491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:40.264076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd3920-64ff-42b0-8789-d8f539aa6d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:40.264770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd3920-64ff-42b0-8789-d8f539aa6d34 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:40.265052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd3920-64ff-42b0-8789-d8f539aa6d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:40.265325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd3920-64ff-42b0-8789-d8f539aa6d34 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:40.266054 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd3920-64ff-42b0-8789-d8f539aa6d34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:40.267645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd3920-64ff-42b0-8789-d8f539aa6d34 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:40.310099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 68/141] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:40:40 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:41.061414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b20c-3485-4f7e-906d-72b212424e7f None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:41.062460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b20c-3485-4f7e-906d-72b212424e7f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:41.062953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b20c-3485-4f7e-906d-72b212424e7f None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:41.064851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 74/142] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:41.070429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9d471-c0e1-45a3-8c96-b22f8ebfc6bd None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:41.070722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9d471-c0e1-45a3-8c96-b22f8ebfc6bd None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:41.071001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9d471-c0e1-45a3-8c96-b22f8ebfc6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:41.091663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-72c9d471-c0e1-45a3-8c96-b22f8ebfc6bd None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:41.092842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-72c9d471-c0e1-45a3-8c96-b22f8ebfc6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:41.181259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 69/143] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:41.218565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2b20c-3485-4f7e-906d-72b212424e7f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:41.327897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:41.344178 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2b20c-3485-4f7e-906d-72b212424e7f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:41.382215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04e405-3a1c-4610-a07a-100c2f1b6560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:41.383034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04e405-3a1c-4610-a07a-100c2f1b6560 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:41.383368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04e405-3a1c-4610-a07a-100c2f1b6560 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:41.383651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04e405-3a1c-4610-a07a-100c2f1b6560 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:41.394600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04e405-3a1c-4610-a07a-100c2f1b6560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-de04e405-3a1c-4610-a07a-100c2f1b6560 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:41.402658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:41.403608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:41.404388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 75/144] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:41.411409 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9d471-c0e1-45a3-8c96-b22f8ebfc6bd None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:41.451756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac08507-3d77-4b9f-a33e-9e6e2e44c7a6 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:41.452379 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac08507-3d77-4b9f-a33e-9e6e2e44c7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:41.452851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac08507-3d77-4b9f-a33e-9e6e2e44c7a6 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:41.454435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 76/145] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:40:41.460165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e301ae4-b04f-42d5-85aa-61be7b99d02d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:41.460910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e301ae4-b04f-42d5-85aa-61be7b99d02d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:41.461392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e301ae4-b04f-42d5-85aa-61be7b99d02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:41.482297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-2e301ae4-b04f-42d5-85aa-61be7b99d02d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:41.483912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2e301ae4-b04f-42d5-85aa-61be7b99d02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:41.608195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9d471-c0e1-45a3-8c96-b22f8ebfc6bd None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:41.622329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 77/146] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:41.634593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:41.659467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ccbb3-799d-4174-93b9-c45b21bfd12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:41.660168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ccbb3-799d-4174-93b9-c45b21bfd12c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:41.660470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ccbb3-799d-4174-93b9-c45b21bfd12c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:41.660748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ccbb3-799d-4174-93b9-c45b21bfd12c None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:41.661573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ccbb3-799d-4174-93b9-c45b21bfd12c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:41.661883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ccbb3-799d-4174-93b9-c45b21bfd12c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:41.663368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ccbb3-799d-4174-93b9-c45b21bfd12c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:41.672729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 70/147] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:41.678808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e301ae4-b04f-42d5-85aa-61be7b99d02d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:41.690178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ccbb3-799d-4174-93b9-c45b21bfd12c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:41.826336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:41.871673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e301ae4-b04f-42d5-85aa-61be7b99d02d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:41.921415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ccbb3-799d-4174-93b9-c45b21bfd12c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:41.934127 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b436b-ade0-4537-a17e-79d89248228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:41.935034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b436b-ade0-4537-a17e-79d89248228d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:41.935339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b436b-ade0-4537-a17e-79d89248228d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:41.935617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b436b-ade0-4537-a17e-79d89248228d None admin] PATH_INFO: `/v3/users/8e6df2e0e74b412988d96dc876c164de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:41.937506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b436b-ade0-4537-a17e-79d89248228d None admin] RBAC: Authorizing `identity:get_user(user_id=8e6df2e0e74b412988d96dc876c164de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:41.939239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b436b-ade0-4537-a17e-79d89248228d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:41.942360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 78/148] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:41 2022] GET /identity/v3/users/8e6df2e0e74b412988d96dc876c164de => generated 289 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:41.948219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b436b-ade0-4537-a17e-79d89248228d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:42.003081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef7f20-3009-454f-9575-05ee9f17b27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:42.005168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef7f20-3009-454f-9575-05ee9f17b27d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:42.008388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef7f20-3009-454f-9575-05ee9f17b27d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:42.008891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef7f20-3009-454f-9575-05ee9f17b27d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:42.035181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef7f20-3009-454f-9575-05ee9f17b27d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-0cef7f20-3009-454f-9575-05ee9f17b27d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:42.044597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:42.045517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:42.046284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:42.055956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 71/149] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:40:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:42.061738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef7f20-3009-454f-9575-05ee9f17b27d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:42.117643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:42.177717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b436b-ade0-4537-a17e-79d89248228d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:42.207222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef7f20-3009-454f-9575-05ee9f17b27d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:42.228632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843dc07-7e80-46d7-84f5-a86a3dc1ac8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:42.229888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843dc07-7e80-46d7-84f5-a86a3dc1ac8b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:42.230363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843dc07-7e80-46d7-84f5-a86a3dc1ac8b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:42.230932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843dc07-7e80-46d7-84f5-a86a3dc1ac8b None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:42.232995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843dc07-7e80-46d7-84f5-a86a3dc1ac8b None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:42.234922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843dc07-7e80-46d7-84f5-a86a3dc1ac8b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:42.237009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 79/150] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:41 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:42.238562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2f65f-7c51-4820-a944-ac0f6414d9af None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:42.239044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2f65f-7c51-4820-a944-ac0f6414d9af None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:42.239606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2f65f-7c51-4820-a944-ac0f6414d9af None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:42.241141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 80/151] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:42.248740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae32a8-b5f6-4ce1-b13e-bd65cb4e4b49 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:42.249010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae32a8-b5f6-4ce1-b13e-bd65cb4e4b49 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:42.249287 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae32a8-b5f6-4ce1-b13e-bd65cb4e4b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:42.252706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed37844-f199-4846-bbf5-756f04f6fc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:42.253397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed37844-f199-4846-bbf5-756f04f6fc68 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:42.253753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed37844-f199-4846-bbf5-756f04f6fc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:42.254028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed37844-f199-4846-bbf5-756f04f6fc68 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:42.254794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed37844-f199-4846-bbf5-756f04f6fc68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:42.255098 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed37844-f199-4846-bbf5-756f04f6fc68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:42.256528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed37844-f199-4846-bbf5-756f04f6fc68 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:42.275888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 72/152] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:40:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:42.276733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed37844-f199-4846-bbf5-756f04f6fc68 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:42.286960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-59ae32a8-b5f6-4ce1-b13e-bd65cb4e4b49 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:42.288223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-59ae32a8-b5f6-4ce1-b13e-bd65cb4e4b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:42.305633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:42.393182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:42.399880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed37844-f199-4846-bbf5-756f04f6fc68 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:42.453723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 81/153] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:42.454528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae32a8-b5f6-4ce1-b13e-bd65cb4e4b49 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:42.465196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f80d7-2716-4be7-adf8-23e332278ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:42.466286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f80d7-2716-4be7-adf8-23e332278ade None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:42.466566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f80d7-2716-4be7-adf8-23e332278ade None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:42.466836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f80d7-2716-4be7-adf8-23e332278ade None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04/users/8e6df2e0e74b412988d96dc876c164de/roles/8f325e9c371744daace594fefd8d1279` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:42.472799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f80d7-2716-4be7-adf8-23e332278ade None admin] RBAC: Authorizing `identity:create_grant(project_id=f35ff6000de749fc927ed896def15c04, user_id=8e6df2e0e74b412988d96dc876c164de, role_id=8f325e9c371744daace594fefd8d1279)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:42.474369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f80d7-2716-4be7-adf8-23e332278ade None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:42.508521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 73/154] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:40:42 2022] PUT /identity/v3/projects/f35ff6000de749fc927ed896def15c04/users/8e6df2e0e74b412988d96dc876c164de/roles/8f325e9c371744daace594fefd8d1279 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:40:42.510039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431c412-7530-4ad7-aa3e-451f5af9f546 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:42.510310 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431c412-7530-4ad7-aa3e-451f5af9f546 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:42.510592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431c412-7530-4ad7-aa3e-451f5af9f546 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:42.514620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 74/155] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:42.514620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431c412-7530-4ad7-aa3e-451f5af9f546 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:42.659544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae32a8-b5f6-4ce1-b13e-bd65cb4e4b49 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:42.717014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431c412-7530-4ad7-aa3e-451f5af9f546 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:42.770602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef989a-3b5d-4ee4-9fa3-f61962de1ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:42.771476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef989a-3b5d-4ee4-9fa3-f61962de1ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:42.771763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef989a-3b5d-4ee4-9fa3-f61962de1ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:42.772142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef989a-3b5d-4ee4-9fa3-f61962de1ff2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:42.779617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaa65a-9c55-4d56-9e82-7ad94ffd9fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:42.780529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa65a-9c55-4d56-9e82-7ad94ffd9fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:42.780806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa65a-9c55-4d56-9e82-7ad94ffd9fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:42.781074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa65a-9c55-4d56-9e82-7ad94ffd9fc8 None admin] PATH_INFO: `/v3/users/b052f32177f94f9ea753f4978551e6df` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:42.786230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaa65a-9c55-4d56-9e82-7ad94ffd9fc8 None admin] RBAC: Authorizing `identity:get_user(user_id=b052f32177f94f9ea753f4978551e6df)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:42.787797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaa65a-9c55-4d56-9e82-7ad94ffd9fc8 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:42.789553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 82/156] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:42 2022] GET /identity/v3/users/b052f32177f94f9ea753f4978551e6df => generated 275 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:42.790857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d95b21-6b28-4400-a478-67c4fef43ee0 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:42.791201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d95b21-6b28-4400-a478-67c4fef43ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:42.791503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d95b21-6b28-4400-a478-67c4fef43ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:42.793164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef989a-3b5d-4ee4-9fa3-f61962de1ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-55ef989a-3b5d-4ee4-9fa3-f61962de1ff2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:42.801654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:42.802769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:42.803666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:42.808622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 75/157] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:42.810296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef989a-3b5d-4ee4-9fa3-f61962de1ff2 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:42.822223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-66d95b21-6b28-4400-a478-67c4fef43ee0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:42.823543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-66d95b21-6b28-4400-a478-67c4fef43ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:42.827546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:42.844012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:42.952866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef989a-3b5d-4ee4-9fa3-f61962de1ff2 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:43.001712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 83/158] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:43.002798 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d95b21-6b28-4400-a478-67c4fef43ee0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:43.003899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4ac63-4dff-4ed5-8d08-fbed78a56b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:43.004578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4ac63-4dff-4ed5-8d08-fbed78a56b48 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:43.004864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4ac63-4dff-4ed5-8d08-fbed78a56b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:43.005140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4ac63-4dff-4ed5-8d08-fbed78a56b48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:43.128665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d95b21-6b28-4400-a478-67c4fef43ee0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:43.144258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4ac63-4dff-4ed5-8d08-fbed78a56b48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:43.145794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4ac63-4dff-4ed5-8d08-fbed78a56b48 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:43.167985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f77b1f-4ab8-46f4-ab1d-9bcfb26400fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:43.169000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f77b1f-4ab8-46f4-ab1d-9bcfb26400fe None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:43.169285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f77b1f-4ab8-46f4-ab1d-9bcfb26400fe None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:43.169557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f77b1f-4ab8-46f4-ab1d-9bcfb26400fe None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:43.170321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f77b1f-4ab8-46f4-ab1d-9bcfb26400fe None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:43.170621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f77b1f-4ab8-46f4-ab1d-9bcfb26400fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:43.172203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f77b1f-4ab8-46f4-ab1d-9bcfb26400fe None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:43.182273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:43.182914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 84/159] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:43.188039 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f77b1f-4ab8-46f4-ab1d-9bcfb26400fe None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:43.208529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:43.302277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f77b1f-4ab8-46f4-ab1d-9bcfb26400fe None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:43.348944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21d8bf-e20f-4df9-a1f7-f2c58a4a74b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:43.349797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21d8bf-e20f-4df9-a1f7-f2c58a4a74b8 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:43.350083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21d8bf-e20f-4df9-a1f7-f2c58a4a74b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:43.350356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21d8bf-e20f-4df9-a1f7-f2c58a4a74b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:43.361491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21d8bf-e20f-4df9-a1f7-f2c58a4a74b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-4b21d8bf-e20f-4df9-a1f7-f2c58a4a74b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:43.376705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:43.377629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:43.378393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 85/160] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:43.379046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21d8bf-e20f-4df9-a1f7-f2c58a4a74b8 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:43.491691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:43.537350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 76/161] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 05:40:43.538792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50516926-8b8b-4ee5-a6d9-e724d63fc314 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:43.539081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50516926-8b8b-4ee5-a6d9-e724d63fc314 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:43.539383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50516926-8b8b-4ee5-a6d9-e724d63fc314 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:43.541079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 77/162] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:43.544322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50516926-8b8b-4ee5-a6d9-e724d63fc314 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:43.593989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21d8bf-e20f-4df9-a1f7-f2c58a4a74b8 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:43.643885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819465ab-004a-4be4-a0a1-73ed7b69930a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:43.645480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819465ab-004a-4be4-a0a1-73ed7b69930a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:43.646968 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819465ab-004a-4be4-a0a1-73ed7b69930a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:43.647503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819465ab-004a-4be4-a0a1-73ed7b69930a None admin] PATH_INFO: `/v3/users/b052f32177f94f9ea753f4978551e6df` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:43.649115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819465ab-004a-4be4-a0a1-73ed7b69930a None admin] RBAC: Authorizing `identity:get_user(user_id=b052f32177f94f9ea753f4978551e6df)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:43.651033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819465ab-004a-4be4-a0a1-73ed7b69930a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:43.653949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 86/163] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:43 2022] GET /identity/v3/users/b052f32177f94f9ea753f4978551e6df => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:43.655048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819465ab-004a-4be4-a0a1-73ed7b69930a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:43.792388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50516926-8b8b-4ee5-a6d9-e724d63fc314 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:43.832593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:43.845957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819465ab-004a-4be4-a0a1-73ed7b69930a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:43.873146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f3bfa-7618-4e85-942d-4b3896004495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:43.873827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f3bfa-7618-4e85-942d-4b3896004495 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:43.874214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f3bfa-7618-4e85-942d-4b3896004495 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:43.874607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f3bfa-7618-4e85-942d-4b3896004495 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:43.875404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f3bfa-7618-4e85-942d-4b3896004495 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:43.875707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f3bfa-7618-4e85-942d-4b3896004495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:43.877104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f3bfa-7618-4e85-942d-4b3896004495 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:43.895696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 78/164] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:43.897000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5954e9-a847-4fb1-b4c0-7ad9e7418c24 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:43.897286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5954e9-a847-4fb1-b4c0-7ad9e7418c24 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:43.897558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5954e9-a847-4fb1-b4c0-7ad9e7418c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:43.907846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:43.922364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57e161-b933-4cc4-a3bb-f2377f05bda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:43.923262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57e161-b933-4cc4-a3bb-f2377f05bda1 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:43.923573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57e161-b933-4cc4-a3bb-f2377f05bda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:43.923800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57e161-b933-4cc4-a3bb-f2377f05bda1 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:43.925107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57e161-b933-4cc4-a3bb-f2377f05bda1 None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:43.926593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57e161-b933-4cc4-a3bb-f2377f05bda1 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:43.936839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 87/165] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:43 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:43.937654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57e161-b933-4cc4-a3bb-f2377f05bda1 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:43.938882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ba5954e9-a847-4fb1-b4c0-7ad9e7418c24 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:43.940095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ba5954e9-a847-4fb1-b4c0-7ad9e7418c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:44.125972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 79/166] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:44.126771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5954e9-a847-4fb1-b4c0-7ad9e7418c24 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:44.149607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57e161-b933-4cc4-a3bb-f2377f05bda1 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:44.227151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1cd2a-0fc4-422a-b8ac-9732620f9366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:44.228718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1cd2a-0fc4-422a-b8ac-9732620f9366 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:44.229598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1cd2a-0fc4-422a-b8ac-9732620f9366 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:44.230083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1cd2a-0fc4-422a-b8ac-9732620f9366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:44.323637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:44.335885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1cd2a-0fc4-422a-b8ac-9732620f9366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:44.337458 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1cd2a-0fc4-422a-b8ac-9732620f9366 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:44.454292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5954e9-a847-4fb1-b4c0-7ad9e7418c24 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:44.476327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 88/167] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:44.479613 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:44.482019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1cd2a-0fc4-422a-b8ac-9732620f9366 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:44.527620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4f210-3730-41fd-bc57-2e105866d544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:44.528514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f210-3730-41fd-bc57-2e105866d544 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:44.528789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f210-3730-41fd-bc57-2e105866d544 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:44.529058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f210-3730-41fd-bc57-2e105866d544 None admin] PATH_INFO: `/v3/users/825e62d400e54cc0868912432c257354` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:44.531183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4f210-3730-41fd-bc57-2e105866d544 None admin] RBAC: Authorizing `identity:get_user(user_id=825e62d400e54cc0868912432c257354)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:44.532662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4f210-3730-41fd-bc57-2e105866d544 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:44.534317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 80/168] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:44 2022] GET /identity/v3/users/825e62d400e54cc0868912432c257354 => generated 247 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:44.540455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4f210-3730-41fd-bc57-2e105866d544 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:44.650465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:44.663608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1cd2a-0fc4-422a-b8ac-9732620f9366 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:44.700685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4f210-3730-41fd-bc57-2e105866d544 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:44.730575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde82b4-1a59-47ac-805d-2cbaa953f5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:44.731736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde82b4-1a59-47ac-805d-2cbaa953f5db None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:44.732242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde82b4-1a59-47ac-805d-2cbaa953f5db None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:44.732792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde82b4-1a59-47ac-805d-2cbaa953f5db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:44.733994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde82b4-1a59-47ac-805d-2cbaa953f5db None admin] RBAC: Adding query filter params (role.id=d2054df5a6d24f5d90793b9b81cd4b4b, scope.project.id=f35ff6000de749fc927ed896def15c04, user.id=b052f32177f94f9ea753f4978551e6df) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:44.734447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde82b4-1a59-47ac-805d-2cbaa953f5db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:44.735930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c3572-0813-4637-878a-32c2a3b45dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:44.736447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde82b4-1a59-47ac-805d-2cbaa953f5db None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:44.736943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c3572-0813-4637-878a-32c2a3b45dca None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:44.737318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c3572-0813-4637-878a-32c2a3b45dca None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:44.737687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c3572-0813-4637-878a-32c2a3b45dca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:44.749530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 89/169] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:44 2022] GET /identity/v3/role_assignments?user.id=b052f32177f94f9ea753f4978551e6df&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=d2054df5a6d24f5d90793b9b81cd4b4b => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:44.750418 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde82b4-1a59-47ac-805d-2cbaa953f5db None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:44.762597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c3572-0813-4637-878a-32c2a3b45dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-ea5c3572-0813-4637-878a-32c2a3b45dca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:44.774875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:44.775865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:44.776665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 81/170] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:44.777334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c3572-0813-4637-878a-32c2a3b45dca None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:44.847619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:44.963425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c3572-0813-4637-878a-32c2a3b45dca None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:44.988513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde82b4-1a59-47ac-805d-2cbaa953f5db None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:44.999918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd423527-852a-4666-8fa8-03d8d02675da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:45.000627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd423527-852a-4666-8fa8-03d8d02675da None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.000908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd423527-852a-4666-8fa8-03d8d02675da None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.001230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd423527-852a-4666-8fa8-03d8d02675da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.027098 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53e46-6205-4c22-a894-b0b164ebe432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:45.028123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53e46-6205-4c22-a894-b0b164ebe432 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.028413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53e46-6205-4c22-a894-b0b164ebe432 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.028786 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53e46-6205-4c22-a894-b0b164ebe432 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.029990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b53e46-6205-4c22-a894-b0b164ebe432 None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:45.031628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b53e46-6205-4c22-a894-b0b164ebe432 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:45.033618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 90/171] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:44 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:45.034856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352fd33-d7a6-43b7-958c-8e538a044914 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.035142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352fd33-d7a6-43b7-958c-8e538a044914 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.035442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352fd33-d7a6-43b7-958c-8e538a044914 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.036906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 91/172] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:45.037458 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352fd33-d7a6-43b7-958c-8e538a044914 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:45.074362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:45.082776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd423527-852a-4666-8fa8-03d8d02675da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:45.083593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd423527-852a-4666-8fa8-03d8d02675da None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:45.127074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:45.162676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352fd33-d7a6-43b7-958c-8e538a044914 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:45.200603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa0c3a-732f-4ffe-9178-d97ec26694a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:45.200603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa0c3a-732f-4ffe-9178-d97ec26694a6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.200603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa0c3a-732f-4ffe-9178-d97ec26694a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.200603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa0c3a-732f-4ffe-9178-d97ec26694a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.200997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa0c3a-732f-4ffe-9178-d97ec26694a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:45.200997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa0c3a-732f-4ffe-9178-d97ec26694a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:45.201128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 82/173] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:45.201199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa0c3a-732f-4ffe-9178-d97ec26694a6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:45.201263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c547a8-5d6d-4333-ba0a-62ae408ad48d None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.201263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c547a8-5d6d-4333-ba0a-62ae408ad48d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.202200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c547a8-5d6d-4333-ba0a-62ae408ad48d None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.202896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 83/174] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:45.208090 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffaf004-7a84-45e8-8531-cc4fab57e1a2 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.208372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffaf004-7a84-45e8-8531-cc4fab57e1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.208644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffaf004-7a84-45e8-8531-cc4fab57e1a2 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.214440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 84/176] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:45.214440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffaf004-7a84-45e8-8531-cc4fab57e1a2 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:45.214659 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 92/176] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:45.215360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13851b66-76e5-4816-bf97-8721f9619a08 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.215653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13851b66-76e5-4816-bf97-8721f9619a08 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.215928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13851b66-76e5-4816-bf97-8721f9619a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.232728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-13851b66-76e5-4816-bf97-8721f9619a08 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:45.233939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-13851b66-76e5-4816-bf97-8721f9619a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:45.323356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 93/177] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:45.323356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13851b66-76e5-4816-bf97-8721f9619a08 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:45.334298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffaf004-7a84-45e8-8531-cc4fab57e1a2 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:45.346344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:45.370745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31d01b-ac57-4bd8-9f08-2906daf34ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:45.371541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31d01b-ac57-4bd8-9f08-2906daf34ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.371932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31d01b-ac57-4bd8-9f08-2906daf34ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.372273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31d01b-ac57-4bd8-9f08-2906daf34ff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.373237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31d01b-ac57-4bd8-9f08-2906daf34ff9 None admin] RBAC: Adding query filter params (role.id=d0a32856510a480a897677657ace20de, scope.project.id=f35ff6000de749fc927ed896def15c04, user.id=b052f32177f94f9ea753f4978551e6df) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:45.373810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31d01b-ac57-4bd8-9f08-2906daf34ff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:45.375092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31d01b-ac57-4bd8-9f08-2906daf34ff9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:45.391284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 85/178] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:45 2022] GET /identity/v3/role_assignments?user.id=b052f32177f94f9ea753f4978551e6df&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=d0a32856510a480a897677657ace20de => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:45.391284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be83f7f-3551-4e9f-8b89-bccf47f08b6b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.391284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be83f7f-3551-4e9f-8b89-bccf47f08b6b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.391284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be83f7f-3551-4e9f-8b89-bccf47f08b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.395894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:45.409167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-7be83f7f-3551-4e9f-8b89-bccf47f08b6b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:45.410484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7be83f7f-3551-4e9f-8b89-bccf47f08b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:45.458273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13851b66-76e5-4816-bf97-8721f9619a08 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:45.487554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 86/179] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:45.496318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be83f7f-3551-4e9f-8b89-bccf47f08b6b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:45.496959 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83761e-3c00-4f4d-9474-e581eb9644b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:45.497843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83761e-3c00-4f4d-9474-e581eb9644b5 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.498117 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83761e-3c00-4f4d-9474-e581eb9644b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.498385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83761e-3c00-4f4d-9474-e581eb9644b5 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.501998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83761e-3c00-4f4d-9474-e581eb9644b5 None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:45.501998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83761e-3c00-4f4d-9474-e581eb9644b5 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:45.508728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 94/180] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:45 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:45.510132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b4f7d-acc8-4ad1-afb2-18bd5b07a2e8 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.510421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b4f7d-acc8-4ad1-afb2-18bd5b07a2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.510693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b4f7d-acc8-4ad1-afb2-18bd5b07a2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.533997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-1f9b4f7d-acc8-4ad1-afb2-18bd5b07a2e8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:45.535335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1f9b4f7d-acc8-4ad1-afb2-18bd5b07a2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:45.621207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 95/181] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:45.621680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b4f7d-acc8-4ad1-afb2-18bd5b07a2e8 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:45.641542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be83f7f-3551-4e9f-8b89-bccf47f08b6b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:45.652613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:45.675492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:45.682926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72bc6b-4d34-4f38-8216-70b6365708e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:45.683862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72bc6b-4d34-4f38-8216-70b6365708e9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.684144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72bc6b-4d34-4f38-8216-70b6365708e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.684436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72bc6b-4d34-4f38-8216-70b6365708e9 None admin] PATH_INFO: `/v3/users/b052f32177f94f9ea753f4978551e6df` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.685735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72bc6b-4d34-4f38-8216-70b6365708e9 None admin] RBAC: Authorizing `identity:get_user(user_id=b052f32177f94f9ea753f4978551e6df)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:45.687201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72bc6b-4d34-4f38-8216-70b6365708e9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:45.688910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 87/182] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:45 2022] GET /identity/v3/users/b052f32177f94f9ea753f4978551e6df => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:45.690113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180a4dc-086b-47b3-800e-d5d198bae8e8 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.690375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180a4dc-086b-47b3-800e-d5d198bae8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.690641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180a4dc-086b-47b3-800e-d5d198bae8e8 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.691917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 88/183] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:45.692613 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180a4dc-086b-47b3-800e-d5d198bae8e8 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:45.736335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b4f7d-acc8-4ad1-afb2-18bd5b07a2e8 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:45.770610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf97b70-3081-4c9c-bd30-0d28678aaa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:45.771445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf97b70-3081-4c9c-bd30-0d28678aaa65 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.772083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf97b70-3081-4c9c-bd30-0d28678aaa65 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.772366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf97b70-3081-4c9c-bd30-0d28678aaa65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.783930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf97b70-3081-4c9c-bd30-0d28678aaa65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-bbf97b70-3081-4c9c-bd30-0d28678aaa65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:45.791703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:45.792606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:45.793404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 96/184] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:40:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:45.794067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf97b70-3081-4c9c-bd30-0d28678aaa65 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:45.799827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180a4dc-086b-47b3-800e-d5d198bae8e8 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:45.832899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef438b-a960-4ee2-b239-cd478a865a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:45.833554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef438b-a960-4ee2-b239-cd478a865a08 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.833835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef438b-a960-4ee2-b239-cd478a865a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.834111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef438b-a960-4ee2-b239-cd478a865a08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.834950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef438b-a960-4ee2-b239-cd478a865a08 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=f35ff6000de749fc927ed896def15c04, user.id=825e62d400e54cc0868912432c257354) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:45.835275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef438b-a960-4ee2-b239-cd478a865a08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:45.836743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef438b-a960-4ee2-b239-cd478a865a08 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:45.842479 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 89/185] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:45 2022] GET /identity/v3/role_assignments?user.id=825e62d400e54cc0868912432c257354&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:45.843236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef438b-a960-4ee2-b239-cd478a865a08 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:45.882573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:45.907781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf97b70-3081-4c9c-bd30-0d28678aaa65 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:45.912314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:45.949394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888b749-97cb-4b78-bc63-0f12ad5b86ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:45.950232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888b749-97cb-4b78-bc63-0f12ad5b86ea None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:45.950513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888b749-97cb-4b78-bc63-0f12ad5b86ea None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:45.950959 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888b749-97cb-4b78-bc63-0f12ad5b86ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:45.956813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef438b-a960-4ee2-b239-cd478a865a08 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:45.963464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888b749-97cb-4b78-bc63-0f12ad5b86ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-e888b749-97cb-4b78-bc63-0f12ad5b86ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:45.971899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:45.972858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:45.973644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 97/186] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:40:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:45.974287 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888b749-97cb-4b78-bc63-0f12ad5b86ea None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:46.000599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b086c0b-8541-4675-9c3d-ed5095854e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:46.000954 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b086c0b-8541-4675-9c3d-ed5095854e5b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.001885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b086c0b-8541-4675-9c3d-ed5095854e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.001885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b086c0b-8541-4675-9c3d-ed5095854e5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.002657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b086c0b-8541-4675-9c3d-ed5095854e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:46.004925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b086c0b-8541-4675-9c3d-ed5095854e5b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:46.006588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 90/187] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:46.007779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2b0b-fec5-44e9-9020-f47f5538300e None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.008061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2b0b-fec5-44e9-9020-f47f5538300e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.008484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2b0b-fec5-44e9-9020-f47f5538300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.025605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-3c8b2b0b-fec5-44e9-9020-f47f5538300e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:46.026799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3c8b2b0b-fec5-44e9-9020-f47f5538300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:46.094634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888b749-97cb-4b78-bc63-0f12ad5b86ea None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:46.101642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 91/188] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:46.102496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2b0b-fec5-44e9-9020-f47f5538300e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:46.129504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10602d-66d2-481f-99e0-307d28d55807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:46.130181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10602d-66d2-481f-99e0-307d28d55807 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.130457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10602d-66d2-481f-99e0-307d28d55807 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.130722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10602d-66d2-481f-99e0-307d28d55807 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.139375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:46.161419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:46.199209 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10602d-66d2-481f-99e0-307d28d55807 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:46.200083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2b0b-fec5-44e9-9020-f47f5538300e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:46.200739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10602d-66d2-481f-99e0-307d28d55807 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:46.236477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726996-d51c-485e-adc0-e2ad8d73c1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:46.237146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726996-d51c-485e-adc0-e2ad8d73c1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.237421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726996-d51c-485e-adc0-e2ad8d73c1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.237686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726996-d51c-485e-adc0-e2ad8d73c1e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.238455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726996-d51c-485e-adc0-e2ad8d73c1e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:46.238750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726996-d51c-485e-adc0-e2ad8d73c1e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:46.240169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726996-d51c-485e-adc0-e2ad8d73c1e9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:46.248202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 92/189] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:46.251758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726996-d51c-485e-adc0-e2ad8d73c1e9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:46.298463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 98/190] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:46.299304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10602d-66d2-481f-99e0-307d28d55807 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:46.338280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:46.349831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726996-d51c-485e-adc0-e2ad8d73c1e9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:46.372108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:46.383477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead506cb-25cc-44fc-b0f7-cb1f89cb2711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:46.384144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead506cb-25cc-44fc-b0f7-cb1f89cb2711 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.384567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead506cb-25cc-44fc-b0f7-cb1f89cb2711 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.384882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead506cb-25cc-44fc-b0f7-cb1f89cb2711 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.385630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead506cb-25cc-44fc-b0f7-cb1f89cb2711 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:46.385933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead506cb-25cc-44fc-b0f7-cb1f89cb2711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:46.387335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead506cb-25cc-44fc-b0f7-cb1f89cb2711 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:46.395504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 93/191] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:40:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:46.396147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead506cb-25cc-44fc-b0f7-cb1f89cb2711 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:46.403653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10602d-66d2-481f-99e0-307d28d55807 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:46.441575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432950f-aaf2-4ba6-ad0f-17b8346d1c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:46.442236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432950f-aaf2-4ba6-ad0f-17b8346d1c61 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.442515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432950f-aaf2-4ba6-ad0f-17b8346d1c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.442788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432950f-aaf2-4ba6-ad0f-17b8346d1c61 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.443538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432950f-aaf2-4ba6-ad0f-17b8346d1c61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:46.445368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432950f-aaf2-4ba6-ad0f-17b8346d1c61 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:46.466791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 99/192] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:40:46 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:46.467636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432950f-aaf2-4ba6-ad0f-17b8346d1c61 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:46.510830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead506cb-25cc-44fc-b0f7-cb1f89cb2711 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:46.550044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd38444-0751-4d66-ace8-cae51b5353a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:46.550894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd38444-0751-4d66-ace8-cae51b5353a7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.551177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd38444-0751-4d66-ace8-cae51b5353a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.551489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd38444-0751-4d66-ace8-cae51b5353a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.567835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd38444-0751-4d66-ace8-cae51b5353a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-8bd38444-0751-4d66-ace8-cae51b5353a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:46.576948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:46.577900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:40:46.578678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:46.579328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 94/193] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:46.579958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd38444-0751-4d66-ace8-cae51b5353a7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:46.588665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:46.600442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432950f-aaf2-4ba6-ad0f-17b8346d1c61 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:46.641258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:46.641845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb3cb3-90f1-4e3b-b201-98a30fb9d975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:46.642751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb3cb3-90f1-4e3b-b201-98a30fb9d975 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.643029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb3cb3-90f1-4e3b-b201-98a30fb9d975 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.643333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb3cb3-90f1-4e3b-b201-98a30fb9d975 None admin] PATH_INFO: `/v3/users/129fa63f85c54334b414297cb0eca69c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.658507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb3cb3-90f1-4e3b-b201-98a30fb9d975 None admin] RBAC: Authorizing `identity:get_user(user_id=129fa63f85c54334b414297cb0eca69c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:46.659948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb3cb3-90f1-4e3b-b201-98a30fb9d975 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:46.661863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 100/194] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:46 2022] GET /identity/v3/users/129fa63f85c54334b414297cb0eca69c => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:46.662468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb3cb3-90f1-4e3b-b201-98a30fb9d975 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:46.717743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd38444-0751-4d66-ace8-cae51b5353a7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:46.759644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef529ebd-d532-4358-8314-70751243e568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:46.760896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef529ebd-d532-4358-8314-70751243e568 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.761314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef529ebd-d532-4358-8314-70751243e568 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.761723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef529ebd-d532-4358-8314-70751243e568 None admin] PATH_INFO: `/v3/projects/e1c18199281c403490a472221374dbac` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.763864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef529ebd-d532-4358-8314-70751243e568 None admin] RBAC: Authorizing `identity:get_project(project_id=e1c18199281c403490a472221374dbac)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:46.765486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef529ebd-d532-4358-8314-70751243e568 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:46.767415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 95/195] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:46 2022] GET /identity/v3/projects/e1c18199281c403490a472221374dbac => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:46.768082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef529ebd-d532-4358-8314-70751243e568 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:46.786280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb3cb3-90f1-4e3b-b201-98a30fb9d975 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:46.828025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891e918-2392-480d-ae03-689ce3987234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:46.829124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891e918-2392-480d-ae03-689ce3987234 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.829596 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891e918-2392-480d-ae03-689ce3987234 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.829932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891e918-2392-480d-ae03-689ce3987234 None admin] PATH_INFO: `/v3/users/129fa63f85c54334b414297cb0eca69c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.831565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891e918-2392-480d-ae03-689ce3987234 None admin] RBAC: Authorizing `identity:get_user(user_id=129fa63f85c54334b414297cb0eca69c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:46.833603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891e918-2392-480d-ae03-689ce3987234 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:46.835825 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 101/196] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:46 2022] GET /identity/v3/users/129fa63f85c54334b414297cb0eca69c => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:46.836573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891e918-2392-480d-ae03-689ce3987234 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:46.864878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:46.902182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef529ebd-d532-4358-8314-70751243e568 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:46.920456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:46.946616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5dfca-b4b5-4c0f-b9e7-c41e1591609c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:46.947643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5dfca-b4b5-4c0f-b9e7-c41e1591609c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:46.948103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5dfca-b4b5-4c0f-b9e7-c41e1591609c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:46.948546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5dfca-b4b5-4c0f-b9e7-c41e1591609c None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:46.949917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5dfca-b4b5-4c0f-b9e7-c41e1591609c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:46.950438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5dfca-b4b5-4c0f-b9e7-c41e1591609c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:46.952417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5dfca-b4b5-4c0f-b9e7-c41e1591609c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:46.964015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891e918-2392-480d-ae03-689ce3987234 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:46.965530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 96/197] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:46.971646 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5dfca-b4b5-4c0f-b9e7-c41e1591609c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:47.008551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c79529-4b65-4177-a89a-5109455a8c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:47.009322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c79529-4b65-4177-a89a-5109455a8c8e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:47.009702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c79529-4b65-4177-a89a-5109455a8c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:47.009996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c79529-4b65-4177-a89a-5109455a8c8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:47.087960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c79529-4b65-4177-a89a-5109455a8c8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:47.089848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c79529-4b65-4177-a89a-5109455a8c8e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:47.093794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5dfca-b4b5-4c0f-b9e7-c41e1591609c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:47.113048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:47.135789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd95a05-7775-493b-910c-628b7bff5a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:47.136463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd95a05-7775-493b-910c-628b7bff5a47 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:47.137097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd95a05-7775-493b-910c-628b7bff5a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:47.137371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd95a05-7775-493b-910c-628b7bff5a47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:47.138215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd95a05-7775-493b-910c-628b7bff5a47 None admin] RBAC: Adding query filter params (role.id=d0a32856510a480a897677657ace20de, scope.project.id=e1c18199281c403490a472221374dbac, user.id=b052f32177f94f9ea753f4978551e6df) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:47.138545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd95a05-7775-493b-910c-628b7bff5a47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:47.140561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd95a05-7775-493b-910c-628b7bff5a47 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:47.148157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 97/198] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:46 2022] GET /identity/v3/role_assignments?user.id=b052f32177f94f9ea753f4978551e6df&scope.project.id=e1c18199281c403490a472221374dbac&role.id=d0a32856510a480a897677657ace20de => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:47.148861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd95a05-7775-493b-910c-628b7bff5a47 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:47.204684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 102/199] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:47.206124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c79529-4b65-4177-a89a-5109455a8c8e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:47.212513 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:47.293878 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd95a05-7775-493b-910c-628b7bff5a47 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:47.320580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c79529-4b65-4177-a89a-5109455a8c8e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:47.340671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7d061-3af5-4efb-8c39-fbc1a3fe5f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:47.341691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7d061-3af5-4efb-8c39-fbc1a3fe5f09 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:47.342000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7d061-3af5-4efb-8c39-fbc1a3fe5f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:47.342278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7d061-3af5-4efb-8c39-fbc1a3fe5f09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:47.345349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:47.386668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee37b0c-e656-4d59-94ce-3dc9c6bbeea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:47.388072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee37b0c-e656-4d59-94ce-3dc9c6bbeea6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:47.388762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee37b0c-e656-4d59-94ce-3dc9c6bbeea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:47.390253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee37b0c-e656-4d59-94ce-3dc9c6bbeea6 None admin] PATH_INFO: `/v3/users/8e6df2e0e74b412988d96dc876c164de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:47.392673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee37b0c-e656-4d59-94ce-3dc9c6bbeea6 None admin] RBAC: Authorizing `identity:get_user(user_id=8e6df2e0e74b412988d96dc876c164de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:47.396722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee37b0c-e656-4d59-94ce-3dc9c6bbeea6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:47.401375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 103/200] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:47 2022] GET /identity/v3/users/8e6df2e0e74b412988d96dc876c164de => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:47.407986 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee37b0c-e656-4d59-94ce-3dc9c6bbeea6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:47.458963 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7d061-3af5-4efb-8c39-fbc1a3fe5f09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:47.460503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7d061-3af5-4efb-8c39-fbc1a3fe5f09 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:47.508876 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:47.542166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee37b0c-e656-4d59-94ce-3dc9c6bbeea6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:47.586017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcc6b7-db82-4731-afb0-4fa98b8ddbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:47.586726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 98/201] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:47.586948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcc6b7-db82-4731-afb0-4fa98b8ddbfa None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:47.587772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcc6b7-db82-4731-afb0-4fa98b8ddbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:47.587772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcc6b7-db82-4731-afb0-4fa98b8ddbfa None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:47.588941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7d061-3af5-4efb-8c39-fbc1a3fe5f09 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:47.592275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcc6b7-db82-4731-afb0-4fa98b8ddbfa None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:47.593791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcc6b7-db82-4731-afb0-4fa98b8ddbfa None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:47.595633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 104/202] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:47 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:47.601178 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcc6b7-db82-4731-afb0-4fa98b8ddbfa None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:47.684928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:47.720893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7d061-3af5-4efb-8c39-fbc1a3fe5f09 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:47.743308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcc6b7-db82-4731-afb0-4fa98b8ddbfa None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:47.758915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444798-d6f2-4456-87fb-67ec12237ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:47.760039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444798-d6f2-4456-87fb-67ec12237ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:47.760492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444798-d6f2-4456-87fb-67ec12237ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:47.760915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444798-d6f2-4456-87fb-67ec12237ed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:47.772675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:47.783128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9d675-8dd1-4dfa-95fd-89fcb200f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:47.784610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9d675-8dd1-4dfa-95fd-89fcb200f656 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:47.784951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9d675-8dd1-4dfa-95fd-89fcb200f656 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:47.785291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9d675-8dd1-4dfa-95fd-89fcb200f656 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:47.786681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9d675-8dd1-4dfa-95fd-89fcb200f656 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:47.789017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9d675-8dd1-4dfa-95fd-89fcb200f656 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:47.796723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 105/203] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:47 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:47.796723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9d675-8dd1-4dfa-95fd-89fcb200f656 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:47.854458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49444798-d6f2-4456-87fb-67ec12237ed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:47.855995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49444798-d6f2-4456-87fb-67ec12237ed9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:47.913317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9d675-8dd1-4dfa-95fd-89fcb200f656 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:47.945731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:47.959894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71734beb-d591-4025-bb15-7f95bf4380be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:47.960554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71734beb-d591-4025-bb15-7f95bf4380be None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:47.960829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71734beb-d591-4025-bb15-7f95bf4380be None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:47.961102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71734beb-d591-4025-bb15-7f95bf4380be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:47.962193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71734beb-d591-4025-bb15-7f95bf4380be None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=81bfce5971f14bb58ff604d83072bd74, user.id=129fa63f85c54334b414297cb0eca69c) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:47.962526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71734beb-d591-4025-bb15-7f95bf4380be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:47.964798 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71734beb-d591-4025-bb15-7f95bf4380be None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:47.970491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 99/204] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:47.971687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444798-d6f2-4456-87fb-67ec12237ed9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:47.972052 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 106/205] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:47 2022] GET /identity/v3/role_assignments?user.id=129fa63f85c54334b414297cb0eca69c&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:47.980374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71734beb-d591-4025-bb15-7f95bf4380be None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:48.020149 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:48.086120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444798-d6f2-4456-87fb-67ec12237ed9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:48.126181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a6c8-4c07-41b6-9925-075d5e47d835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:48.127228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a6c8-4c07-41b6-9925-075d5e47d835 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:48.127735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a6c8-4c07-41b6-9925-075d5e47d835 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:48.128203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a6c8-4c07-41b6-9925-075d5e47d835 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:48.129275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364a6c8-4c07-41b6-9925-075d5e47d835 None admin] RBAC: Adding query filter params (role.id=d2054df5a6d24f5d90793b9b81cd4b4b, scope.project.id=81bfce5971f14bb58ff604d83072bd74, user.id=129fa63f85c54334b414297cb0eca69c) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:48.129785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364a6c8-4c07-41b6-9925-075d5e47d835 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:48.131498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364a6c8-4c07-41b6-9925-075d5e47d835 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:48.138780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 100/206] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:47 2022] GET /identity/v3/role_assignments?user.id=129fa63f85c54334b414297cb0eca69c&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=d2054df5a6d24f5d90793b9b81cd4b4b => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:48.170989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71734beb-d591-4025-bb15-7f95bf4380be None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:48.218472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d0b8d-0fe1-4703-b600-40ae889fa202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:48.219770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d0b8d-0fe1-4703-b600-40ae889fa202 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:48.220259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d0b8d-0fe1-4703-b600-40ae889fa202 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:48.220731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d0b8d-0fe1-4703-b600-40ae889fa202 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:48.222352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d0b8d-0fe1-4703-b600-40ae889fa202 None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:48.224218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d0b8d-0fe1-4703-b600-40ae889fa202 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:48.226643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 107/207] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:47 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:48.232614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a6c8-4c07-41b6-9925-075d5e47d835 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:48.348667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a6c8-4c07-41b6-9925-075d5e47d835 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:48.364376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:48.398006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a572d-df7a-477e-a5df-919180138c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:48.398681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a572d-df7a-477e-a5df-919180138c78 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:48.398966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a572d-df7a-477e-a5df-919180138c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:48.399229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a572d-df7a-477e-a5df-919180138c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:48.400238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a572d-df7a-477e-a5df-919180138c78 None admin] RBAC: Adding query filter params (role.id=8f325e9c371744daace594fefd8d1279, scope.project.id=f35ff6000de749fc927ed896def15c04, user.id=8e6df2e0e74b412988d96dc876c164de) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:48.400624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a572d-df7a-477e-a5df-919180138c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:48.402103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a572d-df7a-477e-a5df-919180138c78 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:48.409149 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 101/208] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:48 2022] GET /identity/v3/role_assignments?user.id=8e6df2e0e74b412988d96dc876c164de&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=8f325e9c371744daace594fefd8d1279 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:48.803502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dcf81-e93d-4827-89bd-afa575abd05d None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:48.804017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dcf81-e93d-4827-89bd-afa575abd05d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:48.804017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dcf81-e93d-4827-89bd-afa575abd05d None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:48.805519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 108/209] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:48.812069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973aaaa-ba7b-474c-81d7-42301c3039db None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:48.812521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973aaaa-ba7b-474c-81d7-42301c3039db None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:48.812670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973aaaa-ba7b-474c-81d7-42301c3039db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:48.830903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c973aaaa-ba7b-474c-81d7-42301c3039db None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:48.832132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c973aaaa-ba7b-474c-81d7-42301c3039db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:48.900305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e43a82-7fc5-4af1-bdf3-9c3c3122b14e None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:48.900927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e43a82-7fc5-4af1-bdf3-9c3c3122b14e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:48.901437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e43a82-7fc5-4af1-bdf3-9c3c3122b14e None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:48.903139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 109/210] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:48.907365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 102/211] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:48.909024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20a23b-f30d-434d-b450-a569439c66f1 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:48.909374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20a23b-f30d-434d-b450-a569439c66f1 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:48.909719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20a23b-f30d-434d-b450-a569439c66f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:48.929084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ee20a23b-f30d-434d-b450-a569439c66f1 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:48.929262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973aaaa-ba7b-474c-81d7-42301c3039db None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:48.930520 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ee20a23b-f30d-434d-b450-a569439c66f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:48.994262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:49.030814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:49.034572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973aaaa-ba7b-474c-81d7-42301c3039db None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:49.036732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 110/212] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:49.068408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20a23b-f30d-434d-b450-a569439c66f1 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:49.076206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191c52-aa7a-48a1-8a9d-8131854266ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:49.077098 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191c52-aa7a-48a1-8a9d-8131854266ac None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:49.077477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191c52-aa7a-48a1-8a9d-8131854266ac None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:49.077867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191c52-aa7a-48a1-8a9d-8131854266ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:49.079126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48191c52-aa7a-48a1-8a9d-8131854266ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:49.080857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48191c52-aa7a-48a1-8a9d-8131854266ac None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:49.082964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 103/213] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:49.088573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191c52-aa7a-48a1-8a9d-8131854266ac None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:49.169697 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20a23b-f30d-434d-b450-a569439c66f1 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:49.196430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191c52-aa7a-48a1-8a9d-8131854266ac None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:49.204382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac47e7-63fb-4c22-8aef-7d15139ca874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:49.205189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac47e7-63fb-4c22-8aef-7d15139ca874 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:49.205464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac47e7-63fb-4c22-8aef-7d15139ca874 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:49.205734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac47e7-63fb-4c22-8aef-7d15139ca874 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:49.216402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac47e7-63fb-4c22-8aef-7d15139ca874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-91ac47e7-63fb-4c22-8aef-7d15139ca874 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:49.224057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:49.226121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:49.228141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:49.231175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 111/214] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:40:49.232083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac47e7-63fb-4c22-8aef-7d15139ca874 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:49.235029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7811-a63c-49b5-aa0e-f3ee12541d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:49.235993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7811-a63c-49b5-aa0e-f3ee12541d65 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:49.236418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7811-a63c-49b5-aa0e-f3ee12541d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:49.236855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7811-a63c-49b5-aa0e-f3ee12541d65 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:49.237782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c7811-a63c-49b5-aa0e-f3ee12541d65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:49.239461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c7811-a63c-49b5-aa0e-f3ee12541d65 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:49.262721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 104/215] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:40:49 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:49.284229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:49.347901 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac47e7-63fb-4c22-8aef-7d15139ca874 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:49.385680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe51c7-af40-4cd9-a212-899e90c2137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:49.386373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe51c7-af40-4cd9-a212-899e90c2137e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:49.386648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe51c7-af40-4cd9-a212-899e90c2137e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:49.386914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe51c7-af40-4cd9-a212-899e90c2137e None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:49.387796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe51c7-af40-4cd9-a212-899e90c2137e None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:49.388098 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe51c7-af40-4cd9-a212-899e90c2137e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:49.389511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe51c7-af40-4cd9-a212-899e90c2137e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:49.403683 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 112/216] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:49.405444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7811-a63c-49b5-aa0e-f3ee12541d65 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:49.432801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:49.524356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7811-a63c-49b5-aa0e-f3ee12541d65 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:49.561873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2a5a8-448a-409f-94f7-d0d572c80f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:49.562776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2a5a8-448a-409f-94f7-d0d572c80f6d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:49.563058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2a5a8-448a-409f-94f7-d0d572c80f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:49.563361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2a5a8-448a-409f-94f7-d0d572c80f6d None admin] PATH_INFO: `/v3/users/a81722949005436eacf772c6c6983363` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:49.577817 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2a5a8-448a-409f-94f7-d0d572c80f6d None admin] RBAC: Authorizing `identity:get_user(user_id=a81722949005436eacf772c6c6983363)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:49.579342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2a5a8-448a-409f-94f7-d0d572c80f6d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:49.581395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 105/217] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:49 2022] GET /identity/v3/users/a81722949005436eacf772c6c6983363 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:49.587232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe51c7-af40-4cd9-a212-899e90c2137e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:49.698488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe51c7-af40-4cd9-a212-899e90c2137e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:49.708326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:49.733879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9f978-fd98-49e3-a26a-54be7bfae9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:49.734588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9f978-fd98-49e3-a26a-54be7bfae9fc None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:49.734873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9f978-fd98-49e3-a26a-54be7bfae9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:49.735149 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9f978-fd98-49e3-a26a-54be7bfae9fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:49.808018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9f978-fd98-49e3-a26a-54be7bfae9fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:49.809508 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9f978-fd98-49e3-a26a-54be7bfae9fc None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:49.928878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 113/218] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:49.934611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2a5a8-448a-409f-94f7-d0d572c80f6d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:50.042064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:50.061202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2a5a8-448a-409f-94f7-d0d572c80f6d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:50.095191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925196a-3014-4e55-964c-6b4281ff170a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:50.096100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925196a-3014-4e55-964c-6b4281ff170a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:50.096379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925196a-3014-4e55-964c-6b4281ff170a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:50.096651 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925196a-3014-4e55-964c-6b4281ff170a None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:50.097839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925196a-3014-4e55-964c-6b4281ff170a None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:50.099330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925196a-3014-4e55-964c-6b4281ff170a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:50.101072 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 106/219] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:49 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:50.106969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9f978-fd98-49e3-a26a-54be7bfae9fc None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:50.119775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:50.258218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9f978-fd98-49e3-a26a-54be7bfae9fc None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:50.312646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23bd43-a5e8-45ac-b5e8-1c48d0acec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:50.313256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23bd43-a5e8-45ac-b5e8-1c48d0acec32 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:50.313533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23bd43-a5e8-45ac-b5e8-1c48d0acec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:50.313811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23bd43-a5e8-45ac-b5e8-1c48d0acec32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:50.314658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23bd43-a5e8-45ac-b5e8-1c48d0acec32 None admin] RBAC: Adding query filter params (role.id=d0a32856510a480a897677657ace20de, scope.project.id=81bfce5971f14bb58ff604d83072bd74, user.id=a81722949005436eacf772c6c6983363) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:50.314966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23bd43-a5e8-45ac-b5e8-1c48d0acec32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:50.316470 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23bd43-a5e8-45ac-b5e8-1c48d0acec32 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:50.323155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 114/220] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:50 2022] GET /identity/v3/role_assignments?user.id=a81722949005436eacf772c6c6983363&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=d0a32856510a480a897677657ace20de => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:50.427266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd344d-adaf-464e-ae03-5b66d28fb36c None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:50.427607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd344d-adaf-464e-ae03-5b66d28fb36c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:50.427877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd344d-adaf-464e-ae03-5b66d28fb36c None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:50.429310 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 107/221] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:50.435428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9b11-c75a-4f25-bb63-3b037436c0d8 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:50.435713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9b11-c75a-4f25-bb63-3b037436c0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:50.435991 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9b11-c75a-4f25-bb63-3b037436c0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:50.460333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-248e9b11-c75a-4f25-bb63-3b037436c0d8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:50.461576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-248e9b11-c75a-4f25-bb63-3b037436c0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:50.552749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 115/222] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:50.571957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd1862-4c78-425d-abf9-957aaf4caaf5 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:50.572196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd1862-4c78-425d-abf9-957aaf4caaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:50.572434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd1862-4c78-425d-abf9-957aaf4caaf5 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:50.573829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 108/223] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:50.579174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c734f-213b-481d-a8f3-119f948c5bb8 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:50.579498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c734f-213b-481d-a8f3-119f948c5bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:50.579778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c734f-213b-481d-a8f3-119f948c5bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:50.585425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd1862-4c78-425d-abf9-957aaf4caaf5 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:50.589461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:50.608492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c40c734f-213b-481d-a8f3-119f948c5bb8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:50.609707 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c40c734f-213b-481d-a8f3-119f948c5bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:50.685342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 116/224] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:50.707700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c734f-213b-481d-a8f3-119f948c5bb8 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:50.783009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd1862-4c78-425d-abf9-957aaf4caaf5 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:50.855863 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f3792-4b02-44ae-905e-78469cfe9cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:50.857041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f3792-4b02-44ae-905e-78469cfe9cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:50.858365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f3792-4b02-44ae-905e-78469cfe9cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:50.859914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f3792-4b02-44ae-905e-78469cfe9cd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:50.862905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c734f-213b-481d-a8f3-119f948c5bb8 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:50.884904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f3792-4b02-44ae-905e-78469cfe9cd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-1b2f3792-4b02-44ae-905e-78469cfe9cd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:50.899810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:50.900831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:50.901611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:50.909293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 109/225] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:40:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:40:50.925346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:50.930030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f3792-4b02-44ae-905e-78469cfe9cd3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:50.944411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:50.948020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b66cb-3f40-498a-a28a-41b3fc339148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:50.949340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b66cb-3f40-498a-a28a-41b3fc339148 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:50.949625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b66cb-3f40-498a-a28a-41b3fc339148 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:50.950055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b66cb-3f40-498a-a28a-41b3fc339148 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:50.970203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b66cb-3f40-498a-a28a-41b3fc339148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-559b66cb-3f40-498a-a28a-41b3fc339148 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:50.978575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:50.979477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:50.980297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 117/226] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:40:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:50.992490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b66cb-3f40-498a-a28a-41b3fc339148 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:51.104302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f3792-4b02-44ae-905e-78469cfe9cd3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:51.138635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dc33c-c615-4efc-ad1b-2e4cb9f21cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:51.139564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dc33c-c615-4efc-ad1b-2e4cb9f21cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:51.140007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dc33c-c615-4efc-ad1b-2e4cb9f21cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:51.140418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dc33c-c615-4efc-ad1b-2e4cb9f21cb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:51.141367 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dc33c-c615-4efc-ad1b-2e4cb9f21cb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:51.141810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dc33c-c615-4efc-ad1b-2e4cb9f21cb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:51.143422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dc33c-c615-4efc-ad1b-2e4cb9f21cb4 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:51.152584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 110/227] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:51.159219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dc33c-c615-4efc-ad1b-2e4cb9f21cb4 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:51.196309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b66cb-3f40-498a-a28a-41b3fc339148 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:51.259396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb5a2-8e29-43be-842f-371e85c03bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:51.260368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb5a2-8e29-43be-842f-371e85c03bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:51.260790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb5a2-8e29-43be-842f-371e85c03bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:51.261200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb5a2-8e29-43be-842f-371e85c03bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:51.262096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffb5a2-8e29-43be-842f-371e85c03bd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:51.262544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffb5a2-8e29-43be-842f-371e85c03bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:51.269147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:51.270229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffb5a2-8e29-43be-842f-371e85c03bd6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:51.286354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 118/228] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:40:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:51.292151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb5a2-8e29-43be-842f-371e85c03bd6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:51.304700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dc33c-c615-4efc-ad1b-2e4cb9f21cb4 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:51.346692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdf8df-6473-40a8-b54c-1b632341b1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:51.347622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdf8df-6473-40a8-b54c-1b632341b1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:51.347908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdf8df-6473-40a8-b54c-1b632341b1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:51.348235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdf8df-6473-40a8-b54c-1b632341b1a6 None admin] PATH_INFO: `/v3/users/129fa63f85c54334b414297cb0eca69c` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:51.351990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdf8df-6473-40a8-b54c-1b632341b1a6 None admin] RBAC: Authorizing `identity:get_user(user_id=129fa63f85c54334b414297cb0eca69c)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:51.353640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdf8df-6473-40a8-b54c-1b632341b1a6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:51.355384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 111/229] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:51 2022] GET /identity/v3/users/129fa63f85c54334b414297cb0eca69c => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:51.360885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdf8df-6473-40a8-b54c-1b632341b1a6 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:51.378577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:51.429262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb5a2-8e29-43be-842f-371e85c03bd6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:51.479196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2009a9-abb8-469f-bab7-c9f49b9e7801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:51.480111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2009a9-abb8-469f-bab7-c9f49b9e7801 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:51.480391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2009a9-abb8-469f-bab7-c9f49b9e7801 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:51.480664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2009a9-abb8-469f-bab7-c9f49b9e7801 None admin] PATH_INFO: `/v3/users/129fa63f85c54334b414297cb0eca69c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:51.486232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2009a9-abb8-469f-bab7-c9f49b9e7801 None admin] RBAC: Authorizing `identity:get_user(user_id=129fa63f85c54334b414297cb0eca69c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:51.487750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2009a9-abb8-469f-bab7-c9f49b9e7801 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:51.489434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 119/230] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:51 2022] GET /identity/v3/users/129fa63f85c54334b414297cb0eca69c => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:51.495732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2009a9-abb8-469f-bab7-c9f49b9e7801 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:51.570628 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdf8df-6473-40a8-b54c-1b632341b1a6 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:51.639937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7038bf4-7a99-41bf-bac8-c674041b1520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:51.640825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7038bf4-7a99-41bf-bac8-c674041b1520 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:51.641106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7038bf4-7a99-41bf-bac8-c674041b1520 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:51.641376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7038bf4-7a99-41bf-bac8-c674041b1520 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:51.641559 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2009a9-abb8-469f-bab7-c9f49b9e7801 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:51.648012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7038bf4-7a99-41bf-bac8-c674041b1520 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:51.649491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7038bf4-7a99-41bf-bac8-c674041b1520 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:51.653489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 112/231] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:51 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:51.657032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7038bf4-7a99-41bf-bac8-c674041b1520 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:51.673968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:51.706994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:51.728611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a0428-9268-4ad4-a673-bd4de3998678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:51.729499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0428-9268-4ad4-a673-bd4de3998678 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:51.729774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0428-9268-4ad4-a673-bd4de3998678 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:51.730051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0428-9268-4ad4-a673-bd4de3998678 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:51.731700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a0428-9268-4ad4-a673-bd4de3998678 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:51.733185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a0428-9268-4ad4-a673-bd4de3998678 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:51.734920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 120/232] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:51 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:51.741324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a0428-9268-4ad4-a673-bd4de3998678 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:51.848893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7038bf4-7a99-41bf-bac8-c674041b1520 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:51.909949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a0428-9268-4ad4-a673-bd4de3998678 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:51.918637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c5f5a-4914-495a-9a17-19fa696cf2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:51.919621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c5f5a-4914-495a-9a17-19fa696cf2b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:51.919904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c5f5a-4914-495a-9a17-19fa696cf2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:51.920181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c5f5a-4914-495a-9a17-19fa696cf2b5 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74/users/129fa63f85c54334b414297cb0eca69c/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:51.925371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c5f5a-4914-495a-9a17-19fa696cf2b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=81bfce5971f14bb58ff604d83072bd74, user_id=129fa63f85c54334b414297cb0eca69c, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:51.926851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c5f5a-4914-495a-9a17-19fa696cf2b5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:51.954996 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 113/233] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:40:51 2022] PUT /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74/users/129fa63f85c54334b414297cb0eca69c/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:40:51.956371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a132221-93a6-4303-8e17-8bb6aa207a88 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:51.956659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a132221-93a6-4303-8e17-8bb6aa207a88 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:51.956933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a132221-93a6-4303-8e17-8bb6aa207a88 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:51.958420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 114/234] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:51.964778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54d4ec-66a9-42d2-b4d9-85bd4fc46a94 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:51.965093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54d4ec-66a9-42d2-b4d9-85bd4fc46a94 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:51.965333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54d4ec-66a9-42d2-b4d9-85bd4fc46a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:51.975239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d39879-7268-41b8-bb4e-bb24ef5d5a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:51.976730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d39879-7268-41b8-bb4e-bb24ef5d5a6b None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:51.977227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d39879-7268-41b8-bb4e-bb24ef5d5a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:51.977720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d39879-7268-41b8-bb4e-bb24ef5d5a6b None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74/users/129fa63f85c54334b414297cb0eca69c/roles/d2054df5a6d24f5d90793b9b81cd4b4b` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:51.981703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d39879-7268-41b8-bb4e-bb24ef5d5a6b None admin] RBAC: Authorizing `identity:create_grant(project_id=81bfce5971f14bb58ff604d83072bd74, user_id=129fa63f85c54334b414297cb0eca69c, role_id=d2054df5a6d24f5d90793b9b81cd4b4b)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:51.983703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d39879-7268-41b8-bb4e-bb24ef5d5a6b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:52.002847 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-3c54d4ec-66a9-42d2-b4d9-85bd4fc46a94 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:52.004149 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3c54d4ec-66a9-42d2-b4d9-85bd4fc46a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:52.005494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 121/235] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:40:51 2022] PUT /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74/users/129fa63f85c54334b414297cb0eca69c/roles/d2054df5a6d24f5d90793b9b81cd4b4b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:40:52.119083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:52.130979 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 115/236] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:52.133127 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1d5c4-654c-41ff-ad50-15aa4717ef8a None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:52.133702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1d5c4-654c-41ff-ad50-15aa4717ef8a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:52.134176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1d5c4-654c-41ff-ad50-15aa4717ef8a None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:52.135930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 122/237] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:52.142896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271314f-59ed-4267-b0ab-5169e0010f83 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:52.143433 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271314f-59ed-4267-b0ab-5169e0010f83 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:52.143931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271314f-59ed-4267-b0ab-5169e0010f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:52.174872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4271314f-59ed-4267-b0ab-5169e0010f83 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:52.176433 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4271314f-59ed-4267-b0ab-5169e0010f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:52.192482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1d5c4-654c-41ff-ad50-15aa4717ef8a None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:52.279625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:52.288796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 116/238] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:52.322150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271314f-59ed-4267-b0ab-5169e0010f83 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:52.327447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1d5c4-654c-41ff-ad50-15aa4717ef8a None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:52.362643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cd689-4187-4d99-818e-2516888d613f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:52.363562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cd689-4187-4d99-818e-2516888d613f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:52.363843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cd689-4187-4d99-818e-2516888d613f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:52.364119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cd689-4187-4d99-818e-2516888d613f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:52.365304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cd689-4187-4d99-818e-2516888d613f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:52.366899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cd689-4187-4d99-818e-2516888d613f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:52.369066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 123/239] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:52.376528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cd689-4187-4d99-818e-2516888d613f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:52.460946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271314f-59ed-4267-b0ab-5169e0010f83 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:52.510334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:52.526851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295c976-55b3-4f74-b8b7-30902318848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:52.527755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295c976-55b3-4f74-b8b7-30902318848a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:52.528036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295c976-55b3-4f74-b8b7-30902318848a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:52.528354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295c976-55b3-4f74-b8b7-30902318848a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:52.541324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cd689-4187-4d99-818e-2516888d613f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:52.550866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295c976-55b3-4f74-b8b7-30902318848a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-6295c976-55b3-4f74-b8b7-30902318848a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:52.560442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:52.561355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:40:52.562146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:52.563936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 117/240] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:40:52.570149 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295c976-55b3-4f74-b8b7-30902318848a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:52.591610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:52.609611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08bb11-cb6f-4b73-8cf2-58c7cc79b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:52.610352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08bb11-cb6f-4b73-8cf2-58c7cc79b3e0 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:52.610636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08bb11-cb6f-4b73-8cf2-58c7cc79b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:52.610912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08bb11-cb6f-4b73-8cf2-58c7cc79b3e0 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:52.611770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08bb11-cb6f-4b73-8cf2-58c7cc79b3e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:52.613259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08bb11-cb6f-4b73-8cf2-58c7cc79b3e0 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:52.644558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 124/241] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:40:52 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:52.677107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a47360-d0c0-472c-a4e7-28dda9c71ea0 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:52.677420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a47360-d0c0-472c-a4e7-28dda9c71ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:52.677666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a47360-d0c0-472c-a4e7-28dda9c71ea0 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:52.679085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 125/242] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:40:52.690557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c02879-2793-438f-9951-e3dde4a0859a None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:52.690990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c02879-2793-438f-9951-e3dde4a0859a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:52.691425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c02879-2793-438f-9951-e3dde4a0859a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:52.724414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b1c02879-2793-438f-9951-e3dde4a0859a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:52.725665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b1c02879-2793-438f-9951-e3dde4a0859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:52.805771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 126/243] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:52.818170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295c976-55b3-4f74-b8b7-30902318848a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:52.830657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c02879-2793-438f-9951-e3dde4a0859a None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:52.874296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b168f-06a7-41bb-b841-af9d96d5ee66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:52.875944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b168f-06a7-41bb-b841-af9d96d5ee66 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:52.876999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b168f-06a7-41bb-b841-af9d96d5ee66 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:52.877614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b168f-06a7-41bb-b841-af9d96d5ee66 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:52.879148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b168f-06a7-41bb-b841-af9d96d5ee66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:52.880054 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b168f-06a7-41bb-b841-af9d96d5ee66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:52.882929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b168f-06a7-41bb-b841-af9d96d5ee66 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:52.895603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 118/244] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:52.902007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b168f-06a7-41bb-b841-af9d96d5ee66 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:52.958457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:52.972780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c02879-2793-438f-9951-e3dde4a0859a None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:52.975199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:53.014303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38ccf9-c135-4f0d-a6a7-a0f5acb5625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:53.015544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38ccf9-c135-4f0d-a6a7-a0f5acb5625b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:53.016036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38ccf9-c135-4f0d-a6a7-a0f5acb5625b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:53.016511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38ccf9-c135-4f0d-a6a7-a0f5acb5625b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:53.028492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b168f-06a7-41bb-b841-af9d96d5ee66 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:53.031370 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38ccf9-c135-4f0d-a6a7-a0f5acb5625b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-1e38ccf9-c135-4f0d-a6a7-a0f5acb5625b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:53.041193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:53.042081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:53.042866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:53.045322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 127/245] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:53.051627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38ccf9-c135-4f0d-a6a7-a0f5acb5625b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:53.087078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebc315-c495-4fd3-a4e3-becdff15a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:53.088061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc315-c495-4fd3-a4e3-becdff15a016 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:53.089170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc315-c495-4fd3-a4e3-becdff15a016 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:53.089170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc315-c495-4fd3-a4e3-becdff15a016 None admin] PATH_INFO: `/v3/users/bdc4c3789330431e9fe9f687c57e43df` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:53.107434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebc315-c495-4fd3-a4e3-becdff15a016 None admin] RBAC: Authorizing `identity:get_user(user_id=bdc4c3789330431e9fe9f687c57e43df)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:53.108747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebc315-c495-4fd3-a4e3-becdff15a016 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:53.110845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 119/246] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:52 2022] GET /identity/v3/users/bdc4c3789330431e9fe9f687c57e43df => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:53.120204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebc315-c495-4fd3-a4e3-becdff15a016 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:53.190673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38ccf9-c135-4f0d-a6a7-a0f5acb5625b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:53.235918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae436b-e6bc-4006-adf3-85eddbf902c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:53.237125 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae436b-e6bc-4006-adf3-85eddbf902c7 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:53.237952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae436b-e6bc-4006-adf3-85eddbf902c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:53.238663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae436b-e6bc-4006-adf3-85eddbf902c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:53.239593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae436b-e6bc-4006-adf3-85eddbf902c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:53.240046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae436b-e6bc-4006-adf3-85eddbf902c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:53.240626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebc315-c495-4fd3-a4e3-becdff15a016 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:53.241766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae436b-e6bc-4006-adf3-85eddbf902c7 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:53.250801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:53.254233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 128/247] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:53.260467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae436b-e6bc-4006-adf3-85eddbf902c7 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:53.280406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc93e9-5367-436e-9354-caa04303ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:53.281136 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc93e9-5367-436e-9354-caa04303ac89 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:53.281490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc93e9-5367-436e-9354-caa04303ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:53.282018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc93e9-5367-436e-9354-caa04303ac89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:53.293032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:53.357803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc93e9-5367-436e-9354-caa04303ac89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:53.359254 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc93e9-5367-436e-9354-caa04303ac89 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:53.375869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae436b-e6bc-4006-adf3-85eddbf902c7 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:53.412637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece1f5b-4560-41b9-b7f1-0def759e8daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:53.413866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1f5b-4560-41b9-b7f1-0def759e8daf None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:53.414325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1f5b-4560-41b9-b7f1-0def759e8daf None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:53.414773 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1f5b-4560-41b9-b7f1-0def759e8daf None admin] PATH_INFO: `/v3/users/a81722949005436eacf772c6c6983363` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:53.416404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece1f5b-4560-41b9-b7f1-0def759e8daf None admin] RBAC: Authorizing `identity:get_user(user_id=a81722949005436eacf772c6c6983363)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:53.418108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece1f5b-4560-41b9-b7f1-0def759e8daf None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:53.420139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 129/248] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:53 2022] GET /identity/v3/users/a81722949005436eacf772c6c6983363 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:53.425917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece1f5b-4560-41b9-b7f1-0def759e8daf None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:53.467990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 120/249] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:53.473941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc93e9-5367-436e-9354-caa04303ac89 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:53.483632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:53.575672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:53.577706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc93e9-5367-436e-9354-caa04303ac89 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:53.585476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece1f5b-4560-41b9-b7f1-0def759e8daf None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:53.614577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41748594-c470-4455-ab0f-eb410881be31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:53.615403 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748594-c470-4455-ab0f-eb410881be31 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:53.615684 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748594-c470-4455-ab0f-eb410881be31 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:53.615956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748594-c470-4455-ab0f-eb410881be31 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:53.617283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41748594-c470-4455-ab0f-eb410881be31 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:53.618801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41748594-c470-4455-ab0f-eb410881be31 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:53.623434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 121/250] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:53 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:53.627655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41748594-c470-4455-ab0f-eb410881be31 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:53.711624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:53.731978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e2ea2-d7fa-4b66-bb6a-c65a3004c83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:53.732880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e2ea2-d7fa-4b66-bb6a-c65a3004c83c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:53.733012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41748594-c470-4455-ab0f-eb410881be31 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:53.733186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e2ea2-d7fa-4b66-bb6a-c65a3004c83c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:53.733426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e2ea2-d7fa-4b66-bb6a-c65a3004c83c None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:53.739968 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e2ea2-d7fa-4b66-bb6a-c65a3004c83c None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:53.741450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e2ea2-d7fa-4b66-bb6a-c65a3004c83c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:53.743210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 130/251] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:53 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:53.749075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e2ea2-d7fa-4b66-bb6a-c65a3004c83c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:53.769720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39508648-fe40-40e2-8582-99c8221e8248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:53.770392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39508648-fe40-40e2-8582-99c8221e8248 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:53.770742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39508648-fe40-40e2-8582-99c8221e8248 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:53.771452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39508648-fe40-40e2-8582-99c8221e8248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:53.772353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39508648-fe40-40e2-8582-99c8221e8248 None admin] RBAC: Adding query filter params (role.id=8f325e9c371744daace594fefd8d1279, scope.project.id=81bfce5971f14bb58ff604d83072bd74, user.id=bdc4c3789330431e9fe9f687c57e43df) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:53.772661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39508648-fe40-40e2-8582-99c8221e8248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:53.774221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39508648-fe40-40e2-8582-99c8221e8248 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:53.780969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 122/252] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:53 2022] GET /identity/v3/role_assignments?user.id=bdc4c3789330431e9fe9f687c57e43df&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=8f325e9c371744daace594fefd8d1279 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:53.933478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e2ea2-d7fa-4b66-bb6a-c65a3004c83c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:53.982254 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b6844-8290-409f-99cf-aa0f51577417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:53.983169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b6844-8290-409f-99cf-aa0f51577417 None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:53.983476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b6844-8290-409f-99cf-aa0f51577417 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:53.983749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b6844-8290-409f-99cf-aa0f51577417 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74/users/a81722949005436eacf772c6c6983363/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:53.985718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b6844-8290-409f-99cf-aa0f51577417 None admin] RBAC: Authorizing `identity:create_grant(project_id=81bfce5971f14bb58ff604d83072bd74, user_id=a81722949005436eacf772c6c6983363, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:53.987193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b6844-8290-409f-99cf-aa0f51577417 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:54.017020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 131/253] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:40:53 2022] PUT /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74/users/a81722949005436eacf772c6c6983363/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:40:54.585724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f78ac0-6e6f-412f-93b0-e8b74872567f None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:54.586301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f78ac0-6e6f-412f-93b0-e8b74872567f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:54.586418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f78ac0-6e6f-412f-93b0-e8b74872567f None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:54.588257 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 123/254] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:54.595352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4f630-2794-4b76-aef8-750e51c94966 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:54.596060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4f630-2794-4b76-aef8-750e51c94966 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:54.596354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4f630-2794-4b76-aef8-750e51c94966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:54.620233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-e3a4f630-2794-4b76-aef8-750e51c94966 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:54.621448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e3a4f630-2794-4b76-aef8-750e51c94966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:54.643280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:54.702580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 132/255] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:54.730419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f78ac0-6e6f-412f-93b0-e8b74872567f None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:54.858890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f78ac0-6e6f-412f-93b0-e8b74872567f None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:54.901461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a7cfb-4848-43fe-88d3-9c37ef20c2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:54.902347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a7cfb-4848-43fe-88d3-9c37ef20c2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:54.902655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a7cfb-4848-43fe-88d3-9c37ef20c2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:54.902930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a7cfb-4848-43fe-88d3-9c37ef20c2d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:54.914602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:54.923531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a7cfb-4848-43fe-88d3-9c37ef20c2d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-640a7cfb-4848-43fe-88d3-9c37ef20c2d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:54.931476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:54.932381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:40:54.933153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 124/256] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:40:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:54.938306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39424dee-b38f-40a4-976d-7c8e25fbb402 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:54.938591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39424dee-b38f-40a4-976d-7c8e25fbb402 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:54.938865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39424dee-b38f-40a4-976d-7c8e25fbb402 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:54.940285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 133/257] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:54.942479 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a7cfb-4848-43fe-88d3-9c37ef20c2d5 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:54.945534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfe527-ea63-4cec-a880-aa653881599f None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:54.945813 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfe527-ea63-4cec-a880-aa653881599f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:54.946083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfe527-ea63-4cec-a880-aa653881599f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:54.971283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-45cfe527-ea63-4cec-a880-aa653881599f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:54.972510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-45cfe527-ea63-4cec-a880-aa653881599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:55.093510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 134/258] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:40:55.113503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a7cfb-4848-43fe-88d3-9c37ef20c2d5 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:55.128631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfe527-ea63-4cec-a880-aa653881599f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:55.166540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218eede-c5cc-447a-b284-e2879342c3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:55.167665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218eede-c5cc-447a-b284-e2879342c3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:55.168162 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218eede-c5cc-447a-b284-e2879342c3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:55.168622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218eede-c5cc-447a-b284-e2879342c3d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:55.169559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218eede-c5cc-447a-b284-e2879342c3d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:55.170062 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218eede-c5cc-447a-b284-e2879342c3d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:55.171657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218eede-c5cc-447a-b284-e2879342c3d5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:55.185506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 125/259] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:55.191010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218eede-c5cc-447a-b284-e2879342c3d5 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:55.255566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:55.279719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:55.316974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfe527-ea63-4cec-a880-aa653881599f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:55.387407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218eede-c5cc-447a-b284-e2879342c3d5 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:55.392047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93550b-ff44-445c-bb53-92cc31c2e3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:55.394143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93550b-ff44-445c-bb53-92cc31c2e3ea None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:55.394625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93550b-ff44-445c-bb53-92cc31c2e3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:55.395091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93550b-ff44-445c-bb53-92cc31c2e3ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:55.418766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93550b-ff44-445c-bb53-92cc31c2e3ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-ae93550b-ff44-445c-bb53-92cc31c2e3ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:55.428687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:55.429547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:40:55.430425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:55.433282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 135/260] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:40:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:55.441212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93550b-ff44-445c-bb53-92cc31c2e3ea None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:55.506134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93a51b-23c4-463e-beb2-dc9a049879d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:55.507690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93a51b-23c4-463e-beb2-dc9a049879d3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:55.508173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93a51b-23c4-463e-beb2-dc9a049879d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:55.508650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93a51b-23c4-463e-beb2-dc9a049879d3 None admin] PATH_INFO: `/v3/users/129fa63f85c54334b414297cb0eca69c` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:55.518010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93a51b-23c4-463e-beb2-dc9a049879d3 None admin] RBAC: Authorizing `identity:get_user(user_id=129fa63f85c54334b414297cb0eca69c)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:55.518772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93a51b-23c4-463e-beb2-dc9a049879d3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:55.523267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 126/261] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:55 2022] GET /identity/v3/users/129fa63f85c54334b414297cb0eca69c => generated 283 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:55.529059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93a51b-23c4-463e-beb2-dc9a049879d3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:55.651216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93550b-ff44-445c-bb53-92cc31c2e3ea None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:55.732932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9f3cf-e8a7-438e-a8c2-285c8efcc3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:55.733627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9f3cf-e8a7-438e-a8c2-285c8efcc3ab None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:55.733910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9f3cf-e8a7-438e-a8c2-285c8efcc3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:55.734187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9f3cf-e8a7-438e-a8c2-285c8efcc3ab None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:55.734948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9f3cf-e8a7-438e-a8c2-285c8efcc3ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:55.735272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9f3cf-e8a7-438e-a8c2-285c8efcc3ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:55.736698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9f3cf-e8a7-438e-a8c2-285c8efcc3ab None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:55.754315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 136/262] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:40:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:55.762737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9f3cf-e8a7-438e-a8c2-285c8efcc3ab None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:55.770516 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:55.884105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93a51b-23c4-463e-beb2-dc9a049879d3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:55.971726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:55.996681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377f98a-aea6-47a3-958a-b9b954651562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:55.997967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377f98a-aea6-47a3-958a-b9b954651562 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:55.998448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377f98a-aea6-47a3-958a-b9b954651562 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:55.998927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377f98a-aea6-47a3-958a-b9b954651562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:56.003836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9f3cf-e8a7-438e-a8c2-285c8efcc3ab None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:56.087491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e186d7e-d236-40df-ae0b-c43aaaadbb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:56.088796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186d7e-d236-40df-ae0b-c43aaaadbb1c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:56.089269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186d7e-d236-40df-ae0b-c43aaaadbb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:56.089730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186d7e-d236-40df-ae0b-c43aaaadbb1c None admin] PATH_INFO: `/v3/users/129fa63f85c54334b414297cb0eca69c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:56.092217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e186d7e-d236-40df-ae0b-c43aaaadbb1c None admin] RBAC: Authorizing `identity:get_user(user_id=129fa63f85c54334b414297cb0eca69c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:56.093882 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e186d7e-d236-40df-ae0b-c43aaaadbb1c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:56.096725 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 137/263] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:55 2022] GET /identity/v3/users/129fa63f85c54334b414297cb0eca69c => generated 283 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:56.113592 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e186d7e-d236-40df-ae0b-c43aaaadbb1c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:56.163908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377f98a-aea6-47a3-958a-b9b954651562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:56.165403 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377f98a-aea6-47a3-958a-b9b954651562 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:56.228259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:56.274314 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e186d7e-d236-40df-ae0b-c43aaaadbb1c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:56.354439 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 127/264] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 827 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:56.356549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c826d1f-9da2-407d-b304-8c322ccd0102 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:56.356966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c826d1f-9da2-407d-b304-8c322ccd0102 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:56.357376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c826d1f-9da2-407d-b304-8c322ccd0102 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:56.361128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446097b-9b86-49ac-9720-785ae1da0903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:56.362112 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446097b-9b86-49ac-9720-785ae1da0903 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:56.362548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446097b-9b86-49ac-9720-785ae1da0903 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:56.362983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446097b-9b86-49ac-9720-785ae1da0903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:56.379561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 128/265] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:56 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:56.381082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c826d1f-9da2-407d-b304-8c322ccd0102 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:56.475768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:56.589249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446097b-9b86-49ac-9720-785ae1da0903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:56.590743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446097b-9b86-49ac-9720-785ae1da0903 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:56.671607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:56.672897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c826d1f-9da2-407d-b304-8c322ccd0102 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:56.721147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e7180-02e7-4e11-a5cf-d66440406968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:56.722307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e7180-02e7-4e11-a5cf-d66440406968 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:56.722591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e7180-02e7-4e11-a5cf-d66440406968 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:56.722860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e7180-02e7-4e11-a5cf-d66440406968 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:56.724190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e7180-02e7-4e11-a5cf-d66440406968 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:56.724616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 138/266] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 05:40:56.725674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e7180-02e7-4e11-a5cf-d66440406968 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:56.725940 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5e8b6-5b69-46fc-873a-560ff0eeb054 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:56.726228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5e8b6-5b69-46fc-873a-560ff0eeb054 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:56.726501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5e8b6-5b69-46fc-873a-560ff0eeb054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:56.731704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 129/267] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:56 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:56.733030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dd6f4-9fed-4218-9bb5-c560e8f5bd25 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:56.733277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dd6f4-9fed-4218-9bb5-c560e8f5bd25 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:56.733567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dd6f4-9fed-4218-9bb5-c560e8f5bd25 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:56.734996 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 130/268] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:56.744211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dd6f4-9fed-4218-9bb5-c560e8f5bd25 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:56.749437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-58f5e8b6-5b69-46fc-873a-560ff0eeb054 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:56.751016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-58f5e8b6-5b69-46fc-873a-560ff0eeb054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:56.825373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 139/269] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:56.826152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5e8b6-5b69-46fc-873a-560ff0eeb054 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:56.864450 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:56.872262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dd6f4-9fed-4218-9bb5-c560e8f5bd25 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:56.907217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3158fc-e6fd-4a3d-aeb0-32207c94ae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:56.908118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3158fc-e6fd-4a3d-aeb0-32207c94ae0c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:56.908398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3158fc-e6fd-4a3d-aeb0-32207c94ae0c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:56.908671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3158fc-e6fd-4a3d-aeb0-32207c94ae0c None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:56.909858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3158fc-e6fd-4a3d-aeb0-32207c94ae0c None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:56.911360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3158fc-e6fd-4a3d-aeb0-32207c94ae0c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:56.913096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 131/270] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:56 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:56.914250 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857f570-c09b-485e-839b-2e891d693e39 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:56.914533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857f570-c09b-485e-839b-2e891d693e39 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:56.914809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857f570-c09b-485e-839b-2e891d693e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:56.931305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4857f570-c09b-485e-839b-2e891d693e39 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:56.932515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4857f570-c09b-485e-839b-2e891d693e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:56.948626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5e8b6-5b69-46fc-873a-560ff0eeb054 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:56.950478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:56.989475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f721fdc-2329-465f-8213-e90f8dd73c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:56.990284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f721fdc-2329-465f-8213-e90f8dd73c50 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:56.990636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f721fdc-2329-465f-8213-e90f8dd73c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:56.990976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f721fdc-2329-465f-8213-e90f8dd73c50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:56.991967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f721fdc-2329-465f-8213-e90f8dd73c50 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=81bfce5971f14bb58ff604d83072bd74, user.id=129fa63f85c54334b414297cb0eca69c) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:56.992339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f721fdc-2329-465f-8213-e90f8dd73c50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:56.993956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f721fdc-2329-465f-8213-e90f8dd73c50 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:57.006729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 132/271] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:57.015453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 140/272] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:56 2022] GET /identity/v3/role_assignments?user.id=129fa63f85c54334b414297cb0eca69c&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:57.015453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f721fdc-2329-465f-8213-e90f8dd73c50 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:57.016875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab234e-1914-4bb2-bb2c-7ae1c9480acc None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.017169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab234e-1914-4bb2-bb2c-7ae1c9480acc None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.017460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab234e-1914-4bb2-bb2c-7ae1c9480acc None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.019085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 133/273] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:57 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:57.024721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab234e-1914-4bb2-bb2c-7ae1c9480acc None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:57.141590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:57.148520 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f721fdc-2329-465f-8213-e90f8dd73c50 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:57.151092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab234e-1914-4bb2-bb2c-7ae1c9480acc None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:57.187214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aeec3c-72ed-46bc-9ce5-f41ba80962fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:57.188042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeec3c-72ed-46bc-9ce5-f41ba80962fe None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.188335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeec3c-72ed-46bc-9ce5-f41ba80962fe None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.188610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeec3c-72ed-46bc-9ce5-f41ba80962fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.191775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb21b2-f793-4441-b169-f06aac978bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:57.192825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb21b2-f793-4441-b169-f06aac978bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.193304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb21b2-f793-4441-b169-f06aac978bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.193768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb21b2-f793-4441-b169-f06aac978bb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.195226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb21b2-f793-4441-b169-f06aac978bb5 None admin] RBAC: Adding query filter params (role.id=d2054df5a6d24f5d90793b9b81cd4b4b, scope.project.id=81bfce5971f14bb58ff604d83072bd74, user.id=129fa63f85c54334b414297cb0eca69c) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:57.195787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb21b2-f793-4441-b169-f06aac978bb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:57.198086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb21b2-f793-4441-b169-f06aac978bb5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:57.202694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aeec3c-72ed-46bc-9ce5-f41ba80962fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:57.206110 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 134/274] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:57 2022] GET /identity/v3/role_assignments?user.id=129fa63f85c54334b414297cb0eca69c&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=d2054df5a6d24f5d90793b9b81cd4b4b => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:57.207488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f46d5-642d-4a20-9c38-515bb54c1016 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.207779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f46d5-642d-4a20-9c38-515bb54c1016 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.208055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f46d5-642d-4a20-9c38-515bb54c1016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-c8aeec3c-72ed-46bc-9ce5-f41ba80962fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:57.211080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:57.212049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:57.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 141/275] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:57.213649 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aeec3c-72ed-46bc-9ce5-f41ba80962fe None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:57.237538 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-211f46d5-642d-4a20-9c38-515bb54c1016 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:57.238769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-211f46d5-642d-4a20-9c38-515bb54c1016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:57.251084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:57.335006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 135/276] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:57.335754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f46d5-642d-4a20-9c38-515bb54c1016 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:57.348617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aeec3c-72ed-46bc-9ce5-f41ba80962fe None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:57.397512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd23c6-03b3-40d9-947d-7052d62b4801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:57.398685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd23c6-03b3-40d9-947d-7052d62b4801 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.399143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd23c6-03b3-40d9-947d-7052d62b4801 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.399623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd23c6-03b3-40d9-947d-7052d62b4801 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.413399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd23c6-03b3-40d9-947d-7052d62b4801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-26fd23c6-03b3-40d9-947d-7052d62b4801 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:57.423411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:57.424360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:40:57.425173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:57.427360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 142/277] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:57.428461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd23c6-03b3-40d9-947d-7052d62b4801 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:57.448672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:57.465440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f46d5-642d-4a20-9c38-515bb54c1016 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:57.506784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2648e3-ac7f-489a-9d4f-0ad9af02cc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:57.507791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2648e3-ac7f-489a-9d4f-0ad9af02cc30 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.508076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2648e3-ac7f-489a-9d4f-0ad9af02cc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.508376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2648e3-ac7f-489a-9d4f-0ad9af02cc30 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.509106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2648e3-ac7f-489a-9d4f-0ad9af02cc30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:57.509469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2648e3-ac7f-489a-9d4f-0ad9af02cc30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:57.510936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2648e3-ac7f-489a-9d4f-0ad9af02cc30 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:57.522713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 136/278] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:57.525458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2648e3-ac7f-489a-9d4f-0ad9af02cc30 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:57.539625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:57.555029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd23c6-03b3-40d9-947d-7052d62b4801 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:57.587826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17100589-3fa1-4e84-9825-8b2f460d7198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:57.588623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17100589-3fa1-4e84-9825-8b2f460d7198 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.588903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17100589-3fa1-4e84-9825-8b2f460d7198 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.589175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17100589-3fa1-4e84-9825-8b2f460d7198 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.590328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17100589-3fa1-4e84-9825-8b2f460d7198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:57.591865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17100589-3fa1-4e84-9825-8b2f460d7198 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:57.593507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 143/279] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:40:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:57.594670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b81f8-e429-4b31-9103-4b4326812a32 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.594955 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b81f8-e429-4b31-9103-4b4326812a32 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.595233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b81f8-e429-4b31-9103-4b4326812a32 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.600597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 144/280] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:40:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:40:57.600597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b81f8-e429-4b31-9103-4b4326812a32 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:57.630485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2648e3-ac7f-489a-9d4f-0ad9af02cc30 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:57.665339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b69a0e-5e97-43f3-9d6e-ce21700bccd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:57.666071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b69a0e-5e97-43f3-9d6e-ce21700bccd0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.666355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b69a0e-5e97-43f3-9d6e-ce21700bccd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.666625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b69a0e-5e97-43f3-9d6e-ce21700bccd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.667402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b69a0e-5e97-43f3-9d6e-ce21700bccd0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:57.667702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b69a0e-5e97-43f3-9d6e-ce21700bccd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:57.669076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b69a0e-5e97-43f3-9d6e-ce21700bccd0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:57.677179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 137/281] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:57.677894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b69a0e-5e97-43f3-9d6e-ce21700bccd0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:57.692360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:57.698915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b81f8-e429-4b31-9103-4b4326812a32 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:57.735476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb736ff6-a90e-4eef-a991-fbcb30fb3a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:57.736717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb736ff6-a90e-4eef-a991-fbcb30fb3a61 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.737777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb736ff6-a90e-4eef-a991-fbcb30fb3a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.737777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb736ff6-a90e-4eef-a991-fbcb30fb3a61 None admin] PATH_INFO: `/v3/users/2dfab96b2dd142848556344a7a4b5648` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.754842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb736ff6-a90e-4eef-a991-fbcb30fb3a61 None admin] RBAC: Authorizing `identity:get_user(user_id=2dfab96b2dd142848556344a7a4b5648)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:57.756359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb736ff6-a90e-4eef-a991-fbcb30fb3a61 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:57.758060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 145/282] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:57 2022] GET /identity/v3/users/2dfab96b2dd142848556344a7a4b5648 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:57.759334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56154c5-e4c4-4d77-86fa-4a82876c5fa1 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.759625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56154c5-e4c4-4d77-86fa-4a82876c5fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.759900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56154c5-e4c4-4d77-86fa-4a82876c5fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.776325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b56154c5-e4c4-4d77-86fa-4a82876c5fa1 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:40:57.777536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b56154c5-e4c4-4d77-86fa-4a82876c5fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:40:57.787266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:57.792920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b69a0e-5e97-43f3-9d6e-ce21700bccd0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:57.827135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e9235-4d14-4e8c-bf60-7db14cfd2def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:57.827834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e9235-4d14-4e8c-bf60-7db14cfd2def None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:57.828118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e9235-4d14-4e8c-bf60-7db14cfd2def None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:57.828395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e9235-4d14-4e8c-bf60-7db14cfd2def None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:57.829109 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e9235-4d14-4e8c-bf60-7db14cfd2def None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:57.830673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e9235-4d14-4e8c-bf60-7db14cfd2def None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:57.850135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 146/283] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:40:57.863334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 138/284] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:40:57 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:40:57.863334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e9235-4d14-4e8c-bf60-7db14cfd2def None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:57.863601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56154c5-e4c4-4d77-86fa-4a82876c5fa1 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:57.939538 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:57.975456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56154c5-e4c4-4d77-86fa-4a82876c5fa1 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:57.978874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e9235-4d14-4e8c-bf60-7db14cfd2def None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:58.014861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891feee7-23ad-4e9a-a232-05d97fb1ce36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:58.015609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891feee7-23ad-4e9a-a232-05d97fb1ce36 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:58.015894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891feee7-23ad-4e9a-a232-05d97fb1ce36 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:58.016160 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891feee7-23ad-4e9a-a232-05d97fb1ce36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:58.016999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891feee7-23ad-4e9a-a232-05d97fb1ce36 None admin] RBAC: Adding query filter params (role.id=d0a32856510a480a897677657ace20de, user.id=2dfab96b2dd142848556344a7a4b5648, scope.system=all) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:58.017313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891feee7-23ad-4e9a-a232-05d97fb1ce36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:58.018062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac8198-ec39-45db-adce-08cd708b6eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:58.018754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891feee7-23ad-4e9a-a232-05d97fb1ce36 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:58.018954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac8198-ec39-45db-adce-08cd708b6eda None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:58.019229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac8198-ec39-45db-adce-08cd708b6eda None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:58.019527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac8198-ec39-45db-adce-08cd708b6eda None admin] PATH_INFO: `/v3/users/bdc4c3789330431e9fe9f687c57e43df` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:58.020698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac8198-ec39-45db-adce-08cd708b6eda None admin] RBAC: Authorizing `identity:get_user(user_id=bdc4c3789330431e9fe9f687c57e43df)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:58.022143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac8198-ec39-45db-adce-08cd708b6eda None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:58.023804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 147/285] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:57 2022] GET /identity/v3/users/bdc4c3789330431e9fe9f687c57e43df => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:58.024645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac8198-ec39-45db-adce-08cd708b6eda None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:58.027357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 139/286] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jul 4 05:40:57 2022] GET /identity/v3/role_assignments?user.id=2dfab96b2dd142848556344a7a4b5648&scope.system=all&role.id=d0a32856510a480a897677657ace20de => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:58.032580 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891feee7-23ad-4e9a-a232-05d97fb1ce36 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:58.040557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:58.163844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac8198-ec39-45db-adce-08cd708b6eda None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:58.180236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891feee7-23ad-4e9a-a232-05d97fb1ce36 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:58.212115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f011528-0ebd-41c3-865c-e04dd2e627b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:58.213084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f011528-0ebd-41c3-865c-e04dd2e627b0 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:58.213688 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f011528-0ebd-41c3-865c-e04dd2e627b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:58.213977 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f011528-0ebd-41c3-865c-e04dd2e627b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:58.216838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:58.226798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549490c-6f24-4219-9824-2d6f265b8e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:58.228511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549490c-6f24-4219-9824-2d6f265b8e9d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:58.229002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549490c-6f24-4219-9824-2d6f265b8e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:58.229492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549490c-6f24-4219-9824-2d6f265b8e9d None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:58.231060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549490c-6f24-4219-9824-2d6f265b8e9d None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:58.231897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f011528-0ebd-41c3-865c-e04dd2e627b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:40:58.233150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549490c-6f24-4219-9824-2d6f265b8e9d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:58.235434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 140/287] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:58 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:58.240902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549490c-6f24-4219-9824-2d6f265b8e9d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-7f011528-0ebd-41c3-865c-e04dd2e627b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:40:58.241053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:40:58.241856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:40:58.244360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 148/288] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:40:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:40:58.250110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f011528-0ebd-41c3-865c-e04dd2e627b0 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:58.333330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:58.354556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549490c-6f24-4219-9824-2d6f265b8e9d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:58.369879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f011528-0ebd-41c3-865c-e04dd2e627b0 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:58.389225 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d4fea-b203-4fb8-94d6-8f692c0c0dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:58.390128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d4fea-b203-4fb8-94d6-8f692c0c0dec None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:58.390410 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d4fea-b203-4fb8-94d6-8f692c0c0dec None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:58.390684 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d4fea-b203-4fb8-94d6-8f692c0c0dec None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74/users/bdc4c3789330431e9fe9f687c57e43df/roles/8f325e9c371744daace594fefd8d1279` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:58.392735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d4fea-b203-4fb8-94d6-8f692c0c0dec None admin] RBAC: Authorizing `identity:create_grant(project_id=81bfce5971f14bb58ff604d83072bd74, user_id=bdc4c3789330431e9fe9f687c57e43df, role_id=8f325e9c371744daace594fefd8d1279)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:58.394215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d4fea-b203-4fb8-94d6-8f692c0c0dec None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:58.411603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac064606-532f-465c-915d-4e5ddc5e2535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:58.412807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac064606-532f-465c-915d-4e5ddc5e2535 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:58.413038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 141/289] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:40:58 2022] PUT /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74/users/bdc4c3789330431e9fe9f687c57e43df/roles/8f325e9c371744daace594fefd8d1279 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:40:58.413495 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac064606-532f-465c-915d-4e5ddc5e2535 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:58.413925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac064606-532f-465c-915d-4e5ddc5e2535 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:58.414774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac064606-532f-465c-915d-4e5ddc5e2535 None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:58.415142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac064606-532f-465c-915d-4e5ddc5e2535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:58.416798 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac064606-532f-465c-915d-4e5ddc5e2535 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:58.436762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 149/290] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:40:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:40:58.438333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d4fea-b203-4fb8-94d6-8f692c0c0dec None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:58.509855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:58.593540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d4fea-b203-4fb8-94d6-8f692c0c0dec None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:58.637650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951622cb-552d-44cc-a69d-434f6ff82dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:58.638672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951622cb-552d-44cc-a69d-434f6ff82dcb None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:58.638958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951622cb-552d-44cc-a69d-434f6ff82dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:58.639233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951622cb-552d-44cc-a69d-434f6ff82dcb None admin] PATH_INFO: `/v3/users/a81722949005436eacf772c6c6983363` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:58.640528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951622cb-552d-44cc-a69d-434f6ff82dcb None admin] RBAC: Authorizing `identity:get_user(user_id=a81722949005436eacf772c6c6983363)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:58.642010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951622cb-552d-44cc-a69d-434f6ff82dcb None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:58.643756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 142/291] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:40:58 2022] GET /identity/v3/users/a81722949005436eacf772c6c6983363 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:58.649426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac064606-532f-465c-915d-4e5ddc5e2535 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:58.778368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac064606-532f-465c-915d-4e5ddc5e2535 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:58.809105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:58.818680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4164f-c27a-4599-91a4-3945339b3996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:58.820182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4164f-c27a-4599-91a4-3945339b3996 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:58.820672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4164f-c27a-4599-91a4-3945339b3996 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:58.821147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4164f-c27a-4599-91a4-3945339b3996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:58.898159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4164f-c27a-4599-91a4-3945339b3996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:58.900315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4164f-c27a-4599-91a4-3945339b3996 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:59.000242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 150/292] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:40:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:40:59.007363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951622cb-552d-44cc-a69d-434f6ff82dcb None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:59.129479 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951622cb-552d-44cc-a69d-434f6ff82dcb None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:59.156669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:59.165411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7eb60-70c0-43d5-94ea-413eaf093a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:59.167299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7eb60-70c0-43d5-94ea-413eaf093a64 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:59.167605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7eb60-70c0-43d5-94ea-413eaf093a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:59.167887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7eb60-70c0-43d5-94ea-413eaf093a64 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:59.169103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7eb60-70c0-43d5-94ea-413eaf093a64 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:59.170561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7eb60-70c0-43d5-94ea-413eaf093a64 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:59.172633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 143/293] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:40:59 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:40:59.178558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4164f-c27a-4599-91a4-3945339b3996 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:40:59.223334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:40:59.305442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4164f-c27a-4599-91a4-3945339b3996 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:40:59.346719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c3771-eacb-473a-8389-5fa8614b447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:40:59.347412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c3771-eacb-473a-8389-5fa8614b447a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:40:59.347691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c3771-eacb-473a-8389-5fa8614b447a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:40:59.347961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c3771-eacb-473a-8389-5fa8614b447a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:40:59.348799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c3771-eacb-473a-8389-5fa8614b447a None admin] RBAC: Adding query filter params (role.id=d0a32856510a480a897677657ace20de, scope.project.id=81bfce5971f14bb58ff604d83072bd74, user.id=a81722949005436eacf772c6c6983363) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:40:59.349104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c3771-eacb-473a-8389-5fa8614b447a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:40:59.350562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c3771-eacb-473a-8389-5fa8614b447a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:40:59.357272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 151/294] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:40:59 2022] GET /identity/v3/role_assignments?user.id=a81722949005436eacf772c6c6983363&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=d0a32856510a480a897677657ace20de => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:00.238571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdd6a7-e03f-4153-88ce-2133864286da None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.239129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdd6a7-e03f-4153-88ce-2133864286da None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.239129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdd6a7-e03f-4153-88ce-2133864286da None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.240550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 144/295] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:00.245852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdd8b-7c21-491b-ac43-e392399b31f3 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.246248 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdd8b-7c21-491b-ac43-e392399b31f3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.246429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdd8b-7c21-491b-ac43-e392399b31f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.263163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a25cdd8b-7c21-491b-ac43-e392399b31f3 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:00.264410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a25cdd8b-7c21-491b-ac43-e392399b31f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:00.334568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 152/296] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:00.357536 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdd6a7-e03f-4153-88ce-2133864286da None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:00.459743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d67129-32cd-4ed7-b62c-4dafac3626c0 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.460187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d67129-32cd-4ed7-b62c-4dafac3626c0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.460555 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d67129-32cd-4ed7-b62c-4dafac3626c0 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.463363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 153/297] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:41:00.468031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679600a-deff-4539-b4c5-3aa35882c928 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.468405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679600a-deff-4539-b4c5-3aa35882c928 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.468743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679600a-deff-4539-b4c5-3aa35882c928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.468845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdd6a7-e03f-4153-88ce-2133864286da None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:00.485447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-2679600a-deff-4539-b4c5-3aa35882c928 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:00.486733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2679600a-deff-4539-b4c5-3aa35882c928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:00.507379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450f48c-3be6-43e6-826b-1fdd7cd4c8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:00.507379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f48c-3be6-43e6-826b-1fdd7cd4c8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.507379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f48c-3be6-43e6-826b-1fdd7cd4c8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.507379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f48c-3be6-43e6-826b-1fdd7cd4c8a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.509179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:00.520215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450f48c-3be6-43e6-826b-1fdd7cd4c8a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-f450f48c-3be6-43e6-826b-1fdd7cd4c8a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:00.528266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:00.529148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:00.529950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 145/298] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:00.540523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450f48c-3be6-43e6-826b-1fdd7cd4c8a2 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:00.567343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 154/299] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:00.580085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679600a-deff-4539-b4c5-3aa35882c928 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:00.608734 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:00.670501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450f48c-3be6-43e6-826b-1fdd7cd4c8a2 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:00.689092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679600a-deff-4539-b4c5-3aa35882c928 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:00.707480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3e318-fbd8-45cd-9ebd-8aa64f9d93b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:00.708157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3e318-fbd8-45cd-9ebd-8aa64f9d93b6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.708433 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3e318-fbd8-45cd-9ebd-8aa64f9d93b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.708698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3e318-fbd8-45cd-9ebd-8aa64f9d93b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.709445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3e318-fbd8-45cd-9ebd-8aa64f9d93b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:00.709743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3e318-fbd8-45cd-9ebd-8aa64f9d93b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:00.711121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3e318-fbd8-45cd-9ebd-8aa64f9d93b6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:00.720284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 146/300] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:00.721723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a059ef9-decb-4bee-beb7-c97c62d66a3f None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.722047 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a059ef9-decb-4bee-beb7-c97c62d66a3f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.722274 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a059ef9-decb-4bee-beb7-c97c62d66a3f None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.723689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 147/301] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:00.728301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a059ef9-decb-4bee-beb7-c97c62d66a3f None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:00.731525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4490f-da8d-4f18-8250-c2bdfbfbba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:00.732339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4490f-da8d-4f18-8250-c2bdfbfbba67 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.732620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4490f-da8d-4f18-8250-c2bdfbfbba67 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.732891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4490f-da8d-4f18-8250-c2bdfbfbba67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.743483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4490f-da8d-4f18-8250-c2bdfbfbba67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-96e4490f-da8d-4f18-8250-c2bdfbfbba67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:00.751273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:00.752136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:00.752930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 155/302] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:00.754244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dfc25-9d71-4e13-9e1f-130e1410b4fc None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.754244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dfc25-9d71-4e13-9e1f-130e1410b4fc None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.754244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dfc25-9d71-4e13-9e1f-130e1410b4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.771153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-882dfc25-9d71-4e13-9e1f-130e1410b4fc None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:00.772294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-882dfc25-9d71-4e13-9e1f-130e1410b4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:00.784099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:00.842819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a059ef9-decb-4bee-beb7-c97c62d66a3f None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:00.843526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 156/303] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:00.844369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dfc25-9d71-4e13-9e1f-130e1410b4fc None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:00.866124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:00.882851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091c134-3781-4b55-830e-74fed514380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:00.883787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091c134-3781-4b55-830e-74fed514380a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.884077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091c134-3781-4b55-830e-74fed514380a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.884400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091c134-3781-4b55-830e-74fed514380a None admin] PATH_INFO: `/v3/users/6266917d262940bfb039a66b64e1dcf6` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.897536 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091c134-3781-4b55-830e-74fed514380a None admin] RBAC: Authorizing `identity:get_user(user_id=6266917d262940bfb039a66b64e1dcf6)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:00.899002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091c134-3781-4b55-830e-74fed514380a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:00.900799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 148/304] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:41:00 2022] GET /identity/v3/users/6266917d262940bfb039a66b64e1dcf6 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:00.902026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f223ca9-ed76-493d-9b6e-ec83e1519d71 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.902311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f223ca9-ed76-493d-9b6e-ec83e1519d71 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.902587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f223ca9-ed76-493d-9b6e-ec83e1519d71 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.903989 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 149/305] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:00.904537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f223ca9-ed76-493d-9b6e-ec83e1519d71 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:00.943002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dfc25-9d71-4e13-9e1f-130e1410b4fc None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:00.976187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813cd35-f15e-4ef1-9e72-76f07017a856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:00.976862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813cd35-f15e-4ef1-9e72-76f07017a856 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:00.977144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813cd35-f15e-4ef1-9e72-76f07017a856 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:00.977420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813cd35-f15e-4ef1-9e72-76f07017a856 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:00.978203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813cd35-f15e-4ef1-9e72-76f07017a856 None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:00.978514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813cd35-f15e-4ef1-9e72-76f07017a856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:00.979936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813cd35-f15e-4ef1-9e72-76f07017a856 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:00.988041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 157/306] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:00.988839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813cd35-f15e-4ef1-9e72-76f07017a856 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:01.004234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f223ca9-ed76-493d-9b6e-ec83e1519d71 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:01.004531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:01.039057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d2628-33a7-40cd-b47e-4735f716db11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:01.039813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d2628-33a7-40cd-b47e-4735f716db11 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:01.040089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d2628-33a7-40cd-b47e-4735f716db11 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:01.040363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d2628-33a7-40cd-b47e-4735f716db11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:01.041571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d2628-33a7-40cd-b47e-4735f716db11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:01.043040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d2628-33a7-40cd-b47e-4735f716db11 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:01.044698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 150/307] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:01.045839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86e7fd-ade4-43ef-ac66-d7db6186a2e4 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:01.046120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86e7fd-ade4-43ef-ac66-d7db6186a2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:01.046392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86e7fd-ade4-43ef-ac66-d7db6186a2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:01.062131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2c86e7fd-ade4-43ef-ac66-d7db6186a2e4 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:01.063349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2c86e7fd-ade4-43ef-ac66-d7db6186a2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:01.098319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813cd35-f15e-4ef1-9e72-76f07017a856 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:01.101613 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:01.128505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 151/308] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:01.129437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86e7fd-ade4-43ef-ac66-d7db6186a2e4 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:01.132474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58e94c-75ec-49bb-998d-b099aa85ee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:01.133139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58e94c-75ec-49bb-998d-b099aa85ee7c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:01.133416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58e94c-75ec-49bb-998d-b099aa85ee7c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:01.133690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58e94c-75ec-49bb-998d-b099aa85ee7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:01.134561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58e94c-75ec-49bb-998d-b099aa85ee7c None admin] RBAC: Adding query filter params (role.id=8f325e9c371744daace594fefd8d1279, user.id=6266917d262940bfb039a66b64e1dcf6, scope.system=all) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:01.134871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58e94c-75ec-49bb-998d-b099aa85ee7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:01.136357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58e94c-75ec-49bb-998d-b099aa85ee7c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:01.141618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 158/309] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jul 4 05:41:00 2022] GET /identity/v3/role_assignments?user.id=6266917d262940bfb039a66b64e1dcf6&scope.system=all&role.id=8f325e9c371744daace594fefd8d1279 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:01.142366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58e94c-75ec-49bb-998d-b099aa85ee7c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:01.240072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:01.260054 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86e7fd-ade4-43ef-ac66-d7db6186a2e4 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:01.276805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58e94c-75ec-49bb-998d-b099aa85ee7c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:01.298986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bf832-3b95-4b94-89ff-cb98e38f263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:01.299837 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bf832-3b95-4b94-89ff-cb98e38f263c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:01.300114 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bf832-3b95-4b94-89ff-cb98e38f263c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:01.300370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bf832-3b95-4b94-89ff-cb98e38f263c None admin] PATH_INFO: `/v3/users/2dfab96b2dd142848556344a7a4b5648` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:01.302145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bf832-3b95-4b94-89ff-cb98e38f263c None admin] RBAC: Authorizing `identity:get_user(user_id=2dfab96b2dd142848556344a7a4b5648)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:01.303669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bf832-3b95-4b94-89ff-cb98e38f263c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:01.305324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 152/310] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:41:01 2022] GET /identity/v3/users/2dfab96b2dd142848556344a7a4b5648 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:01.306303 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bf832-3b95-4b94-89ff-cb98e38f263c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:01.317382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cafd73-9deb-41ab-bf4c-0a329ee9e916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:01.318035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cafd73-9deb-41ab-bf4c-0a329ee9e916 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:01.318317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cafd73-9deb-41ab-bf4c-0a329ee9e916 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:01.318594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cafd73-9deb-41ab-bf4c-0a329ee9e916 None admin] PATH_INFO: `/v3/groups` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:01.319676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cafd73-9deb-41ab-bf4c-0a329ee9e916 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:01.321129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cafd73-9deb-41ab-bf4c-0a329ee9e916 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:01.339328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 159/311] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Jul 4 05:41:01 2022] POST /identity/v3/groups => generated 229 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:41:01.340521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cafd73-9deb-41ab-bf4c-0a329ee9e916 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:01.366961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:01.431823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bf832-3b95-4b94-89ff-cb98e38f263c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:01.470915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2fe45-e4da-4943-b291-6bf679eeb530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:01.471895 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2fe45-e4da-4943-b291-6bf679eeb530 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:01.472179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2fe45-e4da-4943-b291-6bf679eeb530 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:01.472449 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2fe45-e4da-4943-b291-6bf679eeb530 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:01.485176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2fe45-e4da-4943-b291-6bf679eeb530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:01.491858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cafd73-9deb-41ab-bf4c-0a329ee9e916 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-91e2fe45-e4da-4943-b291-6bf679eeb530 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:01.493471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:01.494423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:41:01.495211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:01.495864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 153/312] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:01.505839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2fe45-e4da-4943-b291-6bf679eeb530 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:01.538249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:01.545160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916d949-afa9-4350-83d7-f29bbba842da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:01.547018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916d949-afa9-4350-83d7-f29bbba842da None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:01.547018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916d949-afa9-4350-83d7-f29bbba842da None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:01.547018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916d949-afa9-4350-83d7-f29bbba842da None admin] PATH_INFO: `/v3/system/users/2dfab96b2dd142848556344a7a4b5648/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:01.550500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916d949-afa9-4350-83d7-f29bbba842da None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2dfab96b2dd142848556344a7a4b5648, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:01.553339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916d949-afa9-4350-83d7-f29bbba842da None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:01.567392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 160/313] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Jul 4 05:41:01 2022] PUT /identity/v3/system/users/2dfab96b2dd142848556344a7a4b5648/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:01.630687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2fe45-e4da-4943-b291-6bf679eeb530 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:01.632855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:01.673597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fadd2-703b-4566-bf77-9aa641d54c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:01.674742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fadd2-703b-4566-bf77-9aa641d54c8c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:01.675357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fadd2-703b-4566-bf77-9aa641d54c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:01.675834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fadd2-703b-4566-bf77-9aa641d54c8c None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:01.676911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fadd2-703b-4566-bf77-9aa641d54c8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:01.677476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fadd2-703b-4566-bf77-9aa641d54c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:01.679318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fadd2-703b-4566-bf77-9aa641d54c8c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:01.690316 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 154/314] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:01.696016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916d949-afa9-4350-83d7-f29bbba842da None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:01.814446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916d949-afa9-4350-83d7-f29bbba842da None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:01.848990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939d172-7448-4298-83a3-ca8268218e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:01.850241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939d172-7448-4298-83a3-ca8268218e2d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:01.850741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939d172-7448-4298-83a3-ca8268218e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:01.851208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939d172-7448-4298-83a3-ca8268218e2d None admin] PATH_INFO: `/v3/users/bdc4c3789330431e9fe9f687c57e43df` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:01.852627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939d172-7448-4298-83a3-ca8268218e2d None admin] RBAC: Authorizing `identity:get_user(user_id=bdc4c3789330431e9fe9f687c57e43df)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:01.854333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939d172-7448-4298-83a3-ca8268218e2d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:01.856641 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 161/315] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:41:01 2022] GET /identity/v3/users/bdc4c3789330431e9fe9f687c57e43df => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:01.863269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fadd2-703b-4566-bf77-9aa641d54c8c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:01.973612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fadd2-703b-4566-bf77-9aa641d54c8c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:02.013636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42d6c5-e3a8-4d6a-9b06-0ac36adb49ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:02.014298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42d6c5-e3a8-4d6a-9b06-0ac36adb49ec None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:02.014580 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42d6c5-e3a8-4d6a-9b06-0ac36adb49ec None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:02.014854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42d6c5-e3a8-4d6a-9b06-0ac36adb49ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:02.047138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:02.085853 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42d6c5-e3a8-4d6a-9b06-0ac36adb49ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:02.087280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42d6c5-e3a8-4d6a-9b06-0ac36adb49ec None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:02.189067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 155/316] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:02.194898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939d172-7448-4298-83a3-ca8268218e2d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:02.258984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:02.300153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939d172-7448-4298-83a3-ca8268218e2d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:02.334705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19329fb2-fa39-431d-bb3e-e088705c8f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:02.335604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19329fb2-fa39-431d-bb3e-e088705c8f70 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:02.335964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19329fb2-fa39-431d-bb3e-e088705c8f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:02.336150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19329fb2-fa39-431d-bb3e-e088705c8f70 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:02.337338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19329fb2-fa39-431d-bb3e-e088705c8f70 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:02.338865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19329fb2-fa39-431d-bb3e-e088705c8f70 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:02.340949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 162/317] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:02 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:02.348400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42d6c5-e3a8-4d6a-9b06-0ac36adb49ec None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:02.409969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:02.447714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42d6c5-e3a8-4d6a-9b06-0ac36adb49ec None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:02.480344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ee8ce-c510-4f1f-8126-e61039972c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:02.481006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ee8ce-c510-4f1f-8126-e61039972c16 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:02.481284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ee8ce-c510-4f1f-8126-e61039972c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:02.481553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ee8ce-c510-4f1f-8126-e61039972c16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:02.482443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ee8ce-c510-4f1f-8126-e61039972c16 None admin] RBAC: Adding query filter params (role.id=8f325e9c371744daace594fefd8d1279, scope.project.id=81bfce5971f14bb58ff604d83072bd74, user.id=bdc4c3789330431e9fe9f687c57e43df) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:02.482753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ee8ce-c510-4f1f-8126-e61039972c16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:02.484320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ee8ce-c510-4f1f-8126-e61039972c16 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:02.494340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 156/318] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:02 2022] GET /identity/v3/role_assignments?user.id=bdc4c3789330431e9fe9f687c57e43df&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=8f325e9c371744daace594fefd8d1279 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:03.216532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272b092-cc53-4774-8998-9a4922706d4a None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.216969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272b092-cc53-4774-8998-9a4922706d4a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.217088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272b092-cc53-4774-8998-9a4922706d4a None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.218476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 163/319] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:03.223976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5f6ed-2b27-4b9d-9c88-b2400d44748d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.224228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5f6ed-2b27-4b9d-9c88-b2400d44748d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.224542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5f6ed-2b27-4b9d-9c88-b2400d44748d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.241863 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-86b5f6ed-2b27-4b9d-9c88-b2400d44748d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:03.243061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-86b5f6ed-2b27-4b9d-9c88-b2400d44748d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:03.317976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 157/320] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:03.340645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272b092-cc53-4774-8998-9a4922706d4a None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:03.446957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272b092-cc53-4774-8998-9a4922706d4a None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:03.477158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:03.483527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1637aac-34a0-412a-b877-58422f347d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:03.484328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1637aac-34a0-412a-b877-58422f347d35 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.484601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1637aac-34a0-412a-b877-58422f347d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.484865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1637aac-34a0-412a-b877-58422f347d35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.495441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1637aac-34a0-412a-b877-58422f347d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-d1637aac-34a0-412a-b877-58422f347d35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:03.503240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:03.504138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:03.510217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 164/321] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:03.513880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5f6ed-2b27-4b9d-9c88-b2400d44748d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:03.541998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb40f70-9b77-448c-8927-a74bcfcfc941 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.542327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb40f70-9b77-448c-8927-a74bcfcfc941 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.542574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb40f70-9b77-448c-8927-a74bcfcfc941 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.543992 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 165/322] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:03.548728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e11bf-0bd1-440f-a908-6401fc51cc3a None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.549009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e11bf-0bd1-440f-a908-6401fc51cc3a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.549290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e11bf-0bd1-440f-a908-6401fc51cc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.566890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-023e11bf-0bd1-440f-a908-6401fc51cc3a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:03.568132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-023e11bf-0bd1-440f-a908-6401fc51cc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:03.577794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:03.621268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5f6ed-2b27-4b9d-9c88-b2400d44748d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:03.638930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 166/323] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:03.656319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422693c-07ea-43e8-9499-9d8d19844b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:03.657009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422693c-07ea-43e8-9499-9d8d19844b31 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.657287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422693c-07ea-43e8-9499-9d8d19844b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.657553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422693c-07ea-43e8-9499-9d8d19844b31 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.658295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422693c-07ea-43e8-9499-9d8d19844b31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:03.658596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422693c-07ea-43e8-9499-9d8d19844b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:03.660024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422693c-07ea-43e8-9499-9d8d19844b31 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:03.661534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e11bf-0bd1-440f-a908-6401fc51cc3a None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:03.668783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 158/324] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:03.674257 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422693c-07ea-43e8-9499-9d8d19844b31 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:03.762521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e11bf-0bd1-440f-a908-6401fc51cc3a None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:03.773312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422693c-07ea-43e8-9499-9d8d19844b31 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:03.783934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:03.793535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:03.797078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4bc69-f972-46f3-8f06-9de6cec469e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:03.798213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4bc69-f972-46f3-8f06-9de6cec469e3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.798487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4bc69-f972-46f3-8f06-9de6cec469e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.798761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4bc69-f972-46f3-8f06-9de6cec469e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.800031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4bc69-f972-46f3-8f06-9de6cec469e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:03.801519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4bc69-f972-46f3-8f06-9de6cec469e3 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:03.803147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 167/325] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:03.804482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d454-a704-412b-90c6-117cf3abd6d2 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.804763 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d454-a704-412b-90c6-117cf3abd6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.805035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d454-a704-412b-90c6-117cf3abd6d2 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.806288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 168/326] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:03.809455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552d454-a704-412b-90c6-117cf3abd6d2 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:03.810855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a933a-644e-42c9-91bf-a8e3dc859407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:03.811788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a933a-644e-42c9-91bf-a8e3dc859407 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.812047 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a933a-644e-42c9-91bf-a8e3dc859407 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.812302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a933a-644e-42c9-91bf-a8e3dc859407 None admin] PATH_INFO: `/v3/users/6266917d262940bfb039a66b64e1dcf6` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.813533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a933a-644e-42c9-91bf-a8e3dc859407 None admin] RBAC: Authorizing `identity:get_user(user_id=6266917d262940bfb039a66b64e1dcf6)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:03.817940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a933a-644e-42c9-91bf-a8e3dc859407 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:03.817940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 159/327] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:41:03 2022] GET /identity/v3/users/6266917d262940bfb039a66b64e1dcf6 => generated 293 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:03.817940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fed00-8b42-4834-837d-e39410745a38 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.817940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fed00-8b42-4834-837d-e39410745a38 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.817940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fed00-8b42-4834-837d-e39410745a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.837050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c29fed00-8b42-4834-837d-e39410745a38 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:03.838474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c29fed00-8b42-4834-837d-e39410745a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:03.905426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 160/328] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:03.906574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fed00-8b42-4834-837d-e39410745a38 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:03.918697 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552d454-a704-412b-90c6-117cf3abd6d2 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:03.954343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf591057-3df5-419b-8f62-ce7b5c86a0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:03.955011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf591057-3df5-419b-8f62-ce7b5c86a0d9 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:03.955314 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf591057-3df5-419b-8f62-ce7b5c86a0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:03.955595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf591057-3df5-419b-8f62-ce7b5c86a0d9 None admin] PATH_INFO: `/v3/groups` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:03.956282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf591057-3df5-419b-8f62-ce7b5c86a0d9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:03.957740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf591057-3df5-419b-8f62-ce7b5c86a0d9 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:03.971855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 169/329] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Jul 4 05:41:03 2022] POST /identity/v3/groups => generated 226 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:41:03.972658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf591057-3df5-419b-8f62-ce7b5c86a0d9 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:04.030084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:04.033463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fed00-8b42-4834-837d-e39410745a38 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:04.053374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:04.073222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675e260-85fc-4d71-962b-f7b197c031dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:04.074432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675e260-85fc-4d71-962b-f7b197c031dd None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:04.074718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675e260-85fc-4d71-962b-f7b197c031dd None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:04.074993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675e260-85fc-4d71-962b-f7b197c031dd None admin] PATH_INFO: `/v3/system/users/6266917d262940bfb039a66b64e1dcf6/roles/8f325e9c371744daace594fefd8d1279` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:04.076582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675e260-85fc-4d71-962b-f7b197c031dd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6266917d262940bfb039a66b64e1dcf6, role_id=8f325e9c371744daace594fefd8d1279)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:04.078522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675e260-85fc-4d71-962b-f7b197c031dd None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:04.099339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 161/330] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Jul 4 05:41:03 2022] PUT /identity/v3/system/users/6266917d262940bfb039a66b64e1dcf6/roles/8f325e9c371744daace594fefd8d1279 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:04.099571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf591057-3df5-419b-8f62-ce7b5c86a0d9 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:04.140614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922ce3c-9f1b-4a33-9da2-d390f9a85ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:04.141445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922ce3c-9f1b-4a33-9da2-d390f9a85ada None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:04.142018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922ce3c-9f1b-4a33-9da2-d390f9a85ada None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:04.142018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922ce3c-9f1b-4a33-9da2-d390f9a85ada None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:04.156663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922ce3c-9f1b-4a33-9da2-d390f9a85ada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-1922ce3c-9f1b-4a33-9da2-d390f9a85ada None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:04.165082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:04.166303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:04.167339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 170/331] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:04.174154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675e260-85fc-4d71-962b-f7b197c031dd None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:04.300230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675e260-85fc-4d71-962b-f7b197c031dd None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:04.340824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dd39a-92c2-4f30-a21d-71536261fb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:04.341568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dd39a-92c2-4f30-a21d-71536261fb6f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:04.341897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dd39a-92c2-4f30-a21d-71536261fb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:04.342174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dd39a-92c2-4f30-a21d-71536261fb6f None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:04.342937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1dd39a-92c2-4f30-a21d-71536261fb6f None admin] RBAC: Adding query filter params (name=member) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:04.343266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1dd39a-92c2-4f30-a21d-71536261fb6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:04.344714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1dd39a-92c2-4f30-a21d-71536261fb6f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:04.355104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 162/332] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:04.360284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922ce3c-9f1b-4a33-9da2-d390f9a85ada None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:04.465785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922ce3c-9f1b-4a33-9da2-d390f9a85ada None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:04.476944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:04.502181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c89c79-cb4f-434d-92dc-0c9428102d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:04.503081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c89c79-cb4f-434d-92dc-0c9428102d42 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:04.503387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c89c79-cb4f-434d-92dc-0c9428102d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:04.503659 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c89c79-cb4f-434d-92dc-0c9428102d42 None admin] PATH_INFO: `/v3/users/2dfab96b2dd142848556344a7a4b5648` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:04.504870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c89c79-cb4f-434d-92dc-0c9428102d42 None admin] RBAC: Authorizing `identity:get_user(user_id=2dfab96b2dd142848556344a7a4b5648)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:04.506340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c89c79-cb4f-434d-92dc-0c9428102d42 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:04.508346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 171/333] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:41:04 2022] GET /identity/v3/users/2dfab96b2dd142848556344a7a4b5648 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:04.513831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dd39a-92c2-4f30-a21d-71536261fb6f None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:04.574989 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:04.644111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dd39a-92c2-4f30-a21d-71536261fb6f None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:04.680068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bf6e9-bde9-4b0c-a732-335e4856afc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:04.680740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf6e9-bde9-4b0c-a732-335e4856afc3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:04.681019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf6e9-bde9-4b0c-a732-335e4856afc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:04.681360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf6e9-bde9-4b0c-a732-335e4856afc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:04.682216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bf6e9-bde9-4b0c-a732-335e4856afc3 None admin] RBAC: Adding query filter params (role.id=d0a32856510a480a897677657ace20de, user.id=2dfab96b2dd142848556344a7a4b5648, scope.system=all) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:04.682531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bf6e9-bde9-4b0c-a732-335e4856afc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:04.684013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bf6e9-bde9-4b0c-a732-335e4856afc3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:04.698667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 163/334] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jul 4 05:41:04 2022] GET /identity/v3/role_assignments?user.id=2dfab96b2dd142848556344a7a4b5648&scope.system=all&role.id=d0a32856510a480a897677657ace20de => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:06.522166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001f13-991a-404f-8e65-4debf795aceb None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:06.523026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001f13-991a-404f-8e65-4debf795aceb None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:06.523532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001f13-991a-404f-8e65-4debf795aceb None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:06.525710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 172/335] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:06.530711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e86da8-06f1-470b-be9c-8a11e1612e19 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:06.531117 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e86da8-06f1-470b-be9c-8a11e1612e19 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:06.531308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e86da8-06f1-470b-be9c-8a11e1612e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:06.556619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-68e86da8-06f1-470b-be9c-8a11e1612e19 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:06.557809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-68e86da8-06f1-470b-be9c-8a11e1612e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:06.614624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12725f55-2413-4576-9aec-7aa262310ea4 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:06.614624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12725f55-2413-4576-9aec-7aa262310ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:06.614624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12725f55-2413-4576-9aec-7aa262310ea4 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:06.615336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 173/336] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:06.622334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fd02d-e5a2-42e2-8dec-09c44cffb517 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:06.622911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fd02d-e5a2-42e2-8dec-09c44cffb517 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:06.623409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fd02d-e5a2-42e2-8dec-09c44cffb517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:06.653645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-8f3fd02d-e5a2-42e2-8dec-09c44cffb517 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:06.655188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8f3fd02d-e5a2-42e2-8dec-09c44cffb517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:06.684617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 164/337] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:06.732745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e86da8-06f1-470b-be9c-8a11e1612e19 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:06.752950 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 174/338] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:06.779385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fd02d-e5a2-42e2-8dec-09c44cffb517 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:06.811631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:06.917462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e86da8-06f1-470b-be9c-8a11e1612e19 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:06.966927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fd02d-e5a2-42e2-8dec-09c44cffb517 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:06.969189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72554f-d2d4-4fef-b4ae-d4f4195f6808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:06.970021 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72554f-d2d4-4fef-b4ae-d4f4195f6808 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:06.970300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72554f-d2d4-4fef-b4ae-d4f4195f6808 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:06.970568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72554f-d2d4-4fef-b4ae-d4f4195f6808 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:06.991636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:06.992877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72554f-d2d4-4fef-b4ae-d4f4195f6808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-3c72554f-d2d4-4fef-b4ae-d4f4195f6808 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:07.005059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:07.006011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:07.007262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 165/339] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:07.009137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bee83-aad0-4e29-9216-54915908810d None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:07.009137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bee83-aad0-4e29-9216-54915908810d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:07.009137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bee83-aad0-4e29-9216-54915908810d None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:07.010664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 166/340] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:07.015235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bee83-aad0-4e29-9216-54915908810d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:07.031363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecccd4-d571-41af-94a3-66b7cb531bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:07.036186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecccd4-d571-41af-94a3-66b7cb531bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:07.036466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecccd4-d571-41af-94a3-66b7cb531bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:07.036739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecccd4-d571-41af-94a3-66b7cb531bc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:07.058785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecccd4-d571-41af-94a3-66b7cb531bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-14ecccd4-d571-41af-94a3-66b7cb531bc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:07.073253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:07.075386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 175/341] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:41:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:07.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fc6f0-4dd8-403e-a17c-0ecb0b1c835d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:07.080275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fc6f0-4dd8-403e-a17c-0ecb0b1c835d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:07.080275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fc6f0-4dd8-403e-a17c-0ecb0b1c835d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:07.111859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d40fc6f0-4dd8-403e-a17c-0ecb0b1c835d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:07.114349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d40fc6f0-4dd8-403e-a17c-0ecb0b1c835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:07.170776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:07.182415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bee83-aad0-4e29-9216-54915908810d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:07.224934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e27817-8a7d-4bf4-8995-098c36466e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:07.225648 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27817-8a7d-4bf4-8995-098c36466e43 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:07.225931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27817-8a7d-4bf4-8995-098c36466e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:07.226205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27817-8a7d-4bf4-8995-098c36466e43 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:07.227013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e27817-8a7d-4bf4-8995-098c36466e43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:07.227358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e27817-8a7d-4bf4-8995-098c36466e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:07.229093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e27817-8a7d-4bf4-8995-098c36466e43 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:07.241693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 167/342] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:07.242561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e27817-8a7d-4bf4-8995-098c36466e43 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:07.261912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 176/343] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:07.262835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fc6f0-4dd8-403e-a17c-0ecb0b1c835d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:07.392134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e27817-8a7d-4bf4-8995-098c36466e43 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:07.441357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:07.448606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fc6f0-4dd8-403e-a17c-0ecb0b1c835d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:07.451324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d9751-ebeb-41b7-9dca-aebfae86e5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:07.452023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d9751-ebeb-41b7-9dca-aebfae86e5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:07.452301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d9751-ebeb-41b7-9dca-aebfae86e5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:07.452567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d9751-ebeb-41b7-9dca-aebfae86e5e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:07.453321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d9751-ebeb-41b7-9dca-aebfae86e5e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:07.453619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d9751-ebeb-41b7-9dca-aebfae86e5e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:07.455113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d9751-ebeb-41b7-9dca-aebfae86e5e0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:07.471335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 168/344] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:07.477128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d9751-ebeb-41b7-9dca-aebfae86e5e0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:07.485817 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a4bae-9b94-46f3-9bdf-9e2ab74cfe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:07.486536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a4bae-9b94-46f3-9bdf-9e2ab74cfe28 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:07.486819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a4bae-9b94-46f3-9bdf-9e2ab74cfe28 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:07.487093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a4bae-9b94-46f3-9bdf-9e2ab74cfe28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:07.521345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:07.580466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a4bae-9b94-46f3-9bdf-9e2ab74cfe28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:07.581960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a4bae-9b94-46f3-9bdf-9e2ab74cfe28 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:07.630588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d9751-ebeb-41b7-9dca-aebfae86e5e0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:07.679893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36af71-9ea8-4ba2-b407-22e9be61f977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:07.681096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36af71-9ea8-4ba2-b407-22e9be61f977 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:07.681945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36af71-9ea8-4ba2-b407-22e9be61f977 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:07.682354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36af71-9ea8-4ba2-b407-22e9be61f977 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:07.696622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36af71-9ea8-4ba2-b407-22e9be61f977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:07.698619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:07.704056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 177/345] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:07.705339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d45fb-e8e1-4019-b459-b9601db4b6a0 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:07.705624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d45fb-e8e1-4019-b459-b9601db4b6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:07.705932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d45fb-e8e1-4019-b459-b9601db4b6a0 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-5c36af71-9ea8-4ba2-b407-22e9be61f977 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:07.711269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:07.712195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 169/346] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:07.713156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36af71-9ea8-4ba2-b407-22e9be61f977 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:07.713888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 178/347] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:07.713888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d45fb-e8e1-4019-b459-b9601db4b6a0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:07.823737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:07.849050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d45fb-e8e1-4019-b459-b9601db4b6a0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:07.858163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36af71-9ea8-4ba2-b407-22e9be61f977 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:07.893639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b21994-3e58-4f80-bf28-c084c827d732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:07.894542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b21994-3e58-4f80-bf28-c084c827d732 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:07.894817 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b21994-3e58-4f80-bf28-c084c827d732 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:07.895086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b21994-3e58-4f80-bf28-c084c827d732 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:07.896303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b21994-3e58-4f80-bf28-c084c827d732 None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:07.897766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b21994-3e58-4f80-bf28-c084c827d732 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:07.899859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 179/348] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:07 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:07.900760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b21994-3e58-4f80-bf28-c084c827d732 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:07.948424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9640d-3dd6-4191-bb51-4b492b257182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:07.949020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9640d-3dd6-4191-bb51-4b492b257182 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:07.949263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9640d-3dd6-4191-bb51-4b492b257182 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:07.949524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9640d-3dd6-4191-bb51-4b492b257182 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:07.989054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:08.028824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b21994-3e58-4f80-bf28-c084c827d732 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:08.031424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9640d-3dd6-4191-bb51-4b492b257182 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:08.032907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9640d-3dd6-4191-bb51-4b492b257182 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:08.068348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04b621-489e-441d-b650-da908305883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:08.069945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04b621-489e-441d-b650-da908305883f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.070470 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04b621-489e-441d-b650-da908305883f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.070963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04b621-489e-441d-b650-da908305883f None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.072064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04b621-489e-441d-b650-da908305883f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:08.072593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04b621-489e-441d-b650-da908305883f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:08.074188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04b621-489e-441d-b650-da908305883f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:08.084541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 180/349] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:08.086179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db7d63-b266-4680-95bb-129296656d1c None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.086687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db7d63-b266-4680-95bb-129296656d1c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.087204 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db7d63-b266-4680-95bb-129296656d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.105397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-94db7d63-b266-4680-95bb-129296656d1c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:08.107091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-94db7d63-b266-4680-95bb-129296656d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:08.121153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:08.149132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 170/350] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:08.150107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9640d-3dd6-4191-bb51-4b492b257182 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:08.192641 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 181/351] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:08.193528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db7d63-b266-4680-95bb-129296656d1c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:08.266432 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:08.283922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9640d-3dd6-4191-bb51-4b492b257182 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:08.327316 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db7d63-b266-4680-95bb-129296656d1c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:08.328229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cd86c-d08b-474d-b3fd-bced923ee98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:08.330473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd86c-d08b-474d-b3fd-bced923ee98f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.330900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd86c-d08b-474d-b3fd-bced923ee98f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.331329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd86c-d08b-474d-b3fd-bced923ee98f None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.339555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cd86c-d08b-474d-b3fd-bced923ee98f None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:08.341685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cd86c-d08b-474d-b3fd-bced923ee98f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:08.343859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 171/352] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:08 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:08.345152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27217-eb23-4cd6-90c7-1d4a26b26b88 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.345504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27217-eb23-4cd6-90c7-1d4a26b26b88 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.345783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27217-eb23-4cd6-90c7-1d4a26b26b88 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.347051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 172/353] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:08.347807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27217-eb23-4cd6-90c7-1d4a26b26b88 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:08.367929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d140fe-be55-4f63-a982-afa8f5807f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:08.369396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d140fe-be55-4f63-a982-afa8f5807f3a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.369896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d140fe-be55-4f63-a982-afa8f5807f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.370355 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d140fe-be55-4f63-a982-afa8f5807f3a None admin] PATH_INFO: `/v3/users/6266917d262940bfb039a66b64e1dcf6` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.372875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d140fe-be55-4f63-a982-afa8f5807f3a None admin] RBAC: Authorizing `identity:get_user(user_id=6266917d262940bfb039a66b64e1dcf6)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:08.376193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d140fe-be55-4f63-a982-afa8f5807f3a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:08.377273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 182/354] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:41:08 2022] GET /identity/v3/users/6266917d262940bfb039a66b64e1dcf6 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:08.382023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d140fe-be55-4f63-a982-afa8f5807f3a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:08.394200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:08.470523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27217-eb23-4cd6-90c7-1d4a26b26b88 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:08.515976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4e168-b8d4-4471-bb72-30cbff76f1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:08.516299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e168-b8d4-4471-bb72-30cbff76f1af None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.516554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e168-b8d4-4471-bb72-30cbff76f1af None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.516778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e168-b8d4-4471-bb72-30cbff76f1af None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.521477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4e168-b8d4-4471-bb72-30cbff76f1af None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:08.521477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4e168-b8d4-4471-bb72-30cbff76f1af None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:08.521970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 173/355] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:08 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:08.522747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4e168-b8d4-4471-bb72-30cbff76f1af None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:08.528894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d140fe-be55-4f63-a982-afa8f5807f3a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:08.565602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:08.583843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8bba3-69f5-4109-b3a2-bcb348978dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:08.585733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8bba3-69f5-4109-b3a2-bcb348978dfa None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.586017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8bba3-69f5-4109-b3a2-bcb348978dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.586288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8bba3-69f5-4109-b3a2-bcb348978dfa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.599942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8bba3-69f5-4109-b3a2-bcb348978dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-e4c8bba3-69f5-4109-b3a2-bcb348978dfa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:08.609256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:08.610138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:08.610941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:08.618369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 183/356] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:08.618369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f02ba-1ea4-4e36-a2bd-bdf1978ec2c3 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.618369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f02ba-1ea4-4e36-a2bd-bdf1978ec2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.618369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f02ba-1ea4-4e36-a2bd-bdf1978ec2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.637094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-be2f02ba-1ea4-4e36-a2bd-bdf1978ec2c3 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:08.638288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-be2f02ba-1ea4-4e36-a2bd-bdf1978ec2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:08.647741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4e168-b8d4-4471-bb72-30cbff76f1af None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:08.657279 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:08.688899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fa30a-79ff-4bd6-b6d5-34a26b7f4773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:08.691357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fa30a-79ff-4bd6-b6d5-34a26b7f4773 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.691357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fa30a-79ff-4bd6-b6d5-34a26b7f4773 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.691357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fa30a-79ff-4bd6-b6d5-34a26b7f4773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.691357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1fa30a-79ff-4bd6-b6d5-34a26b7f4773 None admin] RBAC: Adding query filter params (group.id=0b9ca3f758cc426988f62660078544b3, role.id=d0a32856510a480a897677657ace20de, scope.project.id=f35ff6000de749fc927ed896def15c04) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:08.691357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1fa30a-79ff-4bd6-b6d5-34a26b7f4773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:08.692718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1fa30a-79ff-4bd6-b6d5-34a26b7f4773 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:08.699315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 174/357] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 4 05:41:08 2022] GET /identity/v3/role_assignments?group.id=0b9ca3f758cc426988f62660078544b3&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=d0a32856510a480a897677657ace20de => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:08.700128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fa30a-79ff-4bd6-b6d5-34a26b7f4773 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:08.717333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 184/358] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:08.717788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f02ba-1ea4-4e36-a2bd-bdf1978ec2c3 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:08.839577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fa30a-79ff-4bd6-b6d5-34a26b7f4773 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:08.842288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f02ba-1ea4-4e36-a2bd-bdf1978ec2c3 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:08.863772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:08.881751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071804b-4b36-4ecf-8f43-3770be3f8126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:08.882424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071804b-4b36-4ecf-8f43-3770be3f8126 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.882703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071804b-4b36-4ecf-8f43-3770be3f8126 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.882972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071804b-4b36-4ecf-8f43-3770be3f8126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.883855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071804b-4b36-4ecf-8f43-3770be3f8126 None admin] RBAC: Adding query filter params (role.id=8f325e9c371744daace594fefd8d1279, user.id=6266917d262940bfb039a66b64e1dcf6, scope.system=all) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:08.884168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071804b-4b36-4ecf-8f43-3770be3f8126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:08.885831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181375f-0da3-4d18-993b-b7f0c0b53407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:08.886713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181375f-0da3-4d18-993b-b7f0c0b53407 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.886993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181375f-0da3-4d18-993b-b7f0c0b53407 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.887300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181375f-0da3-4d18-993b-b7f0c0b53407 None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.888532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181375f-0da3-4d18-993b-b7f0c0b53407 None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:08.888772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071804b-4b36-4ecf-8f43-3770be3f8126 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:08.890009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181375f-0da3-4d18-993b-b7f0c0b53407 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:08.892189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 185/359] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:08 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:08.893215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff2114-3311-428b-9af3-3d65b76ab5bb None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:08.893493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff2114-3311-428b-9af3-3d65b76ab5bb None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:08.893764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff2114-3311-428b-9af3-3d65b76ab5bb None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:08.895503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 186/360] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:08.896211 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff2114-3311-428b-9af3-3d65b76ab5bb None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:08.901996 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 175/361] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jul 4 05:41:08 2022] GET /identity/v3/role_assignments?user.id=6266917d262940bfb039a66b64e1dcf6&scope.system=all&role.id=8f325e9c371744daace594fefd8d1279 => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:08.908374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071804b-4b36-4ecf-8f43-3770be3f8126 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:08.953081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:09.028740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff2114-3311-428b-9af3-3d65b76ab5bb None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:09.054375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071804b-4b36-4ecf-8f43-3770be3f8126 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:09.070361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d706e-7e76-4a42-8e73-890dc3492020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:09.071041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d706e-7e76-4a42-8e73-890dc3492020 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:09.071359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d706e-7e76-4a42-8e73-890dc3492020 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:09.071634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d706e-7e76-4a42-8e73-890dc3492020 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:09.072564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d706e-7e76-4a42-8e73-890dc3492020 None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:09.072872 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d706e-7e76-4a42-8e73-890dc3492020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:09.074390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d706e-7e76-4a42-8e73-890dc3492020 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:09.087069 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 187/362] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:09.088737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a51d35-d676-45a1-b7f8-6728ab260cc7 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:09.089022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a51d35-d676-45a1-b7f8-6728ab260cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:09.089290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a51d35-d676-45a1-b7f8-6728ab260cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:09.097915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1068fa6-39f5-4cee-91ca-f95cbe6e363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:09.098676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1068fa6-39f5-4cee-91ca-f95cbe6e363d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:09.098957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1068fa6-39f5-4cee-91ca-f95cbe6e363d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:09.099229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1068fa6-39f5-4cee-91ca-f95cbe6e363d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:09.109544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-17a51d35-d676-45a1-b7f8-6728ab260cc7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:09.111801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-17a51d35-d676-45a1-b7f8-6728ab260cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:09.115724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1068fa6-39f5-4cee-91ca-f95cbe6e363d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-e1068fa6-39f5-4cee-91ca-f95cbe6e363d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:09.124212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:09.125129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:09.125915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:09.126664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 176/363] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:41:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:09.127443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1068fa6-39f5-4cee-91ca-f95cbe6e363d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:09.167671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:09.196076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 188/364] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:09.197181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a51d35-d676-45a1-b7f8-6728ab260cc7 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:09.242983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:09.258430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1068fa6-39f5-4cee-91ca-f95cbe6e363d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:09.297636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821a658-d7ec-44a3-85e6-94245a6a65a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:09.298582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821a658-d7ec-44a3-85e6-94245a6a65a1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:09.298859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821a658-d7ec-44a3-85e6-94245a6a65a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:09.299129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821a658-d7ec-44a3-85e6-94245a6a65a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:09.300911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821a658-d7ec-44a3-85e6-94245a6a65a1 None admin] RBAC: Adding query filter params (group.id=0b9ca3f758cc426988f62660078544b3, role.id=d2054df5a6d24f5d90793b9b81cd4b4b, scope.project.id=f35ff6000de749fc927ed896def15c04) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:09.300911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821a658-d7ec-44a3-85e6-94245a6a65a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:09.302307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821a658-d7ec-44a3-85e6-94245a6a65a1 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:09.312131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 177/365] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 4 05:41:09 2022] GET /identity/v3/role_assignments?group.id=0b9ca3f758cc426988f62660078544b3&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=d2054df5a6d24f5d90793b9b81cd4b4b => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:09.312784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821a658-d7ec-44a3-85e6-94245a6a65a1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:09.321165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a51d35-d676-45a1-b7f8-6728ab260cc7 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:09.371509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77543d01-99bd-4d9a-b9d1-b585d3477208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:09.372999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77543d01-99bd-4d9a-b9d1-b585d3477208 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:09.373507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77543d01-99bd-4d9a-b9d1-b585d3477208 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:09.373978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77543d01-99bd-4d9a-b9d1-b585d3477208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:09.442267 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:09.464957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821a658-d7ec-44a3-85e6-94245a6a65a1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:09.475456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77543d01-99bd-4d9a-b9d1-b585d3477208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:09.477940 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77543d01-99bd-4d9a-b9d1-b585d3477208 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:09.525584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2641e-031a-4a4c-9f66-41156f979884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:09.526277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2641e-031a-4a4c-9f66-41156f979884 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:09.526553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2641e-031a-4a4c-9f66-41156f979884 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:09.526818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2641e-031a-4a4c-9f66-41156f979884 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:09.527612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2641e-031a-4a4c-9f66-41156f979884 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:09.527914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2641e-031a-4a4c-9f66-41156f979884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:09.529295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2641e-031a-4a4c-9f66-41156f979884 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:09.541968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 178/366] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:09.542845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2641e-031a-4a4c-9f66-41156f979884 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:09.568864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:09.639840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 189/367] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:09.640699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77543d01-99bd-4d9a-b9d1-b585d3477208 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:09.655097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2641e-031a-4a4c-9f66-41156f979884 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:09.693045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b52a1-925d-4719-8de5-91e13545202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:09.693950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b52a1-925d-4719-8de5-91e13545202f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:09.694631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b52a1-925d-4719-8de5-91e13545202f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:09.694999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b52a1-925d-4719-8de5-91e13545202f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:09.707577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b52a1-925d-4719-8de5-91e13545202f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-cd9b52a1-925d-4719-8de5-91e13545202f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:09.715884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:09.716777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:09.717577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 179/368] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:09.718233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b52a1-925d-4719-8de5-91e13545202f None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:09.725819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:09.757374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77543d01-99bd-4d9a-b9d1-b585d3477208 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:09.797952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b5ea3-43fa-45de-a469-649e6f748257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:09.798978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5ea3-43fa-45de-a469-649e6f748257 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:09.799501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5ea3-43fa-45de-a469-649e6f748257 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:09.799967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5ea3-43fa-45de-a469-649e6f748257 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:09.803790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:09.830055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b52a1-925d-4719-8de5-91e13545202f None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:09.865691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9f756-be61-423d-b616-140c43555009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:09.866573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9f756-be61-423d-b616-140c43555009 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:09.867000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9f756-be61-423d-b616-140c43555009 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:09.867297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9f756-be61-423d-b616-140c43555009 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:09.868537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9f756-be61-423d-b616-140c43555009 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:09.870278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9f756-be61-423d-b616-140c43555009 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:09.873073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 180/369] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:09 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:09.873832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9f756-be61-423d-b616-140c43555009 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:09.897637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b5ea3-43fa-45de-a469-649e6f748257 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:09.899068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b5ea3-43fa-45de-a469-649e6f748257 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:10.002012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9f756-be61-423d-b616-140c43555009 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:10.005666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:10.037698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 190/370] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:10.038384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b5ea3-43fa-45de-a469-649e6f748257 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:10.040993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f84f25-3c65-4e05-af18-8a17b2ddb979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:10.041779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84f25-3c65-4e05-af18-8a17b2ddb979 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:10.042125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84f25-3c65-4e05-af18-8a17b2ddb979 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:10.042460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84f25-3c65-4e05-af18-8a17b2ddb979 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:10.043343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f84f25-3c65-4e05-af18-8a17b2ddb979 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:10.043652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f84f25-3c65-4e05-af18-8a17b2ddb979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:10.045106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f84f25-3c65-4e05-af18-8a17b2ddb979 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:10.054441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 181/371] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:10.055378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f84f25-3c65-4e05-af18-8a17b2ddb979 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:10.064913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:10.170070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b5ea3-43fa-45de-a469-649e6f748257 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:10.192993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f84f25-3c65-4e05-af18-8a17b2ddb979 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:10.218483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da08ac-5cc2-4e9d-8adf-ea36f2c728e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:10.219378 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da08ac-5cc2-4e9d-8adf-ea36f2c728e4 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:10.219658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da08ac-5cc2-4e9d-8adf-ea36f2c728e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:10.219932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da08ac-5cc2-4e9d-8adf-ea36f2c728e4 None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:10.221104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da08ac-5cc2-4e9d-8adf-ea36f2c728e4 None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:10.222563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da08ac-5cc2-4e9d-8adf-ea36f2c728e4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:10.224617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 191/372] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:10 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:10.225456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da08ac-5cc2-4e9d-8adf-ea36f2c728e4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:10.242696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2f96d-ed80-4866-8fba-34b50b486c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:10.244018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2f96d-ed80-4866-8fba-34b50b486c66 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:10.244767 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2f96d-ed80-4866-8fba-34b50b486c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:10.245056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2f96d-ed80-4866-8fba-34b50b486c66 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:10.246293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2f96d-ed80-4866-8fba-34b50b486c66 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:10.248391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2f96d-ed80-4866-8fba-34b50b486c66 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:10.250866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 182/373] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:10 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:10.251749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2f96d-ed80-4866-8fba-34b50b486c66 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:10.280643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:10.341693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da08ac-5cc2-4e9d-8adf-ea36f2c728e4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:10.354738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:10.379197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3886-f24c-4cf5-8655-a6c6398770a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:10.379893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3886-f24c-4cf5-8655-a6c6398770a9 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:10.380228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3886-f24c-4cf5-8655-a6c6398770a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:10.380500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3886-f24c-4cf5-8655-a6c6398770a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:10.388835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2f96d-ed80-4866-8fba-34b50b486c66 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:10.429934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db2df8-a45a-471d-806b-fe0367b03e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:10.430523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db2df8-a45a-471d-806b-fe0367b03e0e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:10.430808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db2df8-a45a-471d-806b-fe0367b03e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:10.431084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db2df8-a45a-471d-806b-fe0367b03e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:10.432209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db2df8-a45a-471d-806b-fe0367b03e0e None admin] RBAC: Adding query filter params (group.id=0b9ca3f758cc426988f62660078544b3, role.id=d0a32856510a480a897677657ace20de, scope.project.id=81bfce5971f14bb58ff604d83072bd74) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:10.432535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db2df8-a45a-471d-806b-fe0367b03e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:10.434073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db2df8-a45a-471d-806b-fe0367b03e0e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:10.443023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 183/374] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 4 05:41:10 2022] GET /identity/v3/role_assignments?group.id=0b9ca3f758cc426988f62660078544b3&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=d0a32856510a480a897677657ace20de => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:10.443932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db2df8-a45a-471d-806b-fe0367b03e0e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:10.499290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c3886-f24c-4cf5-8655-a6c6398770a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:10.511277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c3886-f24c-4cf5-8655-a6c6398770a9 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:10.573523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db2df8-a45a-471d-806b-fe0367b03e0e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:10.576019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:10.614875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29295154-db22-4120-a0bb-64d77fc10699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:10.615800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29295154-db22-4120-a0bb-64d77fc10699 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:10.616177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29295154-db22-4120-a0bb-64d77fc10699 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:10.616454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29295154-db22-4120-a0bb-64d77fc10699 None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:10.617742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29295154-db22-4120-a0bb-64d77fc10699 None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:10.619308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29295154-db22-4120-a0bb-64d77fc10699 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:10.621515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 184/375] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:10 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:10.635286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29295154-db22-4120-a0bb-64d77fc10699 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:10.643534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:10.649908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 192/376] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:10.660377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3886-f24c-4cf5-8655-a6c6398770a9 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:10.800762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29295154-db22-4120-a0bb-64d77fc10699 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:10.834236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3886-f24c-4cf5-8655-a6c6398770a9 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:10.844276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7db461-3702-40de-9d5d-08356f7574af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:10.844959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db461-3702-40de-9d5d-08356f7574af None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:10.845239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db461-3702-40de-9d5d-08356f7574af None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:10.845511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db461-3702-40de-9d5d-08356f7574af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:10.846352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7db461-3702-40de-9d5d-08356f7574af None admin] RBAC: Adding query filter params (group.id=0b9ca3f758cc426988f62660078544b3, role.id=d2054df5a6d24f5d90793b9b81cd4b4b, scope.project.id=81bfce5971f14bb58ff604d83072bd74) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:10.846828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7db461-3702-40de-9d5d-08356f7574af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:10.848498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7db461-3702-40de-9d5d-08356f7574af None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:10.856958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 185/377] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 4 05:41:10 2022] GET /identity/v3/role_assignments?group.id=0b9ca3f758cc426988f62660078544b3&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=d2054df5a6d24f5d90793b9b81cd4b4b => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:10.879380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022268c5-b36b-424a-bc11-d17f4089bca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:10.880651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022268c5-b36b-424a-bc11-d17f4089bca7 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:10.881125 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022268c5-b36b-424a-bc11-d17f4089bca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:10.881606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022268c5-b36b-424a-bc11-d17f4089bca7 None admin] PATH_INFO: `/v3/projects/8a4d1bfa73f141cd90a91a1d9547a701` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:10.883003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022268c5-b36b-424a-bc11-d17f4089bca7 None admin] RBAC: Authorizing `identity:get_project(project_id=8a4d1bfa73f141cd90a91a1d9547a701)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:10.884953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022268c5-b36b-424a-bc11-d17f4089bca7 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:10.887302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 193/378] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:10 2022] GET /identity/v3/projects/8a4d1bfa73f141cd90a91a1d9547a701 => generated 344 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:10.893818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7db461-3702-40de-9d5d-08356f7574af None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:10.987325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:11.022627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7db461-3702-40de-9d5d-08356f7574af None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:11.062978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caa3f3-7813-4aa1-9d05-afc23e70f987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:11.063815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caa3f3-7813-4aa1-9d05-afc23e70f987 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:11.064094 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caa3f3-7813-4aa1-9d05-afc23e70f987 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:11.064369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caa3f3-7813-4aa1-9d05-afc23e70f987 None admin] PATH_INFO: `/v3/groups/d00b2f341ffd4c80afe10b8079f151e3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:11.069910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caa3f3-7813-4aa1-9d05-afc23e70f987 None admin] RBAC: Authorizing `identity:get_group(group_id=d00b2f341ffd4c80afe10b8079f151e3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:11.071433 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caa3f3-7813-4aa1-9d05-afc23e70f987 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:11.078204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 186/379] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:10 2022] GET /identity/v3/groups/d00b2f341ffd4c80afe10b8079f151e3 => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:11.081634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022268c5-b36b-424a-bc11-d17f4089bca7 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:11.102463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:11.187768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97068451-ce67-4e04-827d-3a6e7d914bc9 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:11.188093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97068451-ce67-4e04-827d-3a6e7d914bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:11.188331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97068451-ce67-4e04-827d-3a6e7d914bc9 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:11.190528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 187/380] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:41:11.198337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9e7c2-f43b-4cfd-a7b0-31f23e8d2412 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:11.198624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9e7c2-f43b-4cfd-a7b0-31f23e8d2412 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:11.198897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9e7c2-f43b-4cfd-a7b0-31f23e8d2412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:11.221649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-64b9e7c2-f43b-4cfd-a7b0-31f23e8d2412 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:11.222852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-64b9e7c2-f43b-4cfd-a7b0-31f23e8d2412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:11.289463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022268c5-b36b-424a-bc11-d17f4089bca7 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:11.340832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 188/381] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 04 05:41:11.348393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce60ec-48ce-4470-a526-289852251aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:11.349062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce60ec-48ce-4470-a526-289852251aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:11.349343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce60ec-48ce-4470-a526-289852251aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:11.349923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce60ec-48ce-4470-a526-289852251aa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:11.351349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce60ec-48ce-4470-a526-289852251aa9 None admin] RBAC: Adding query filter params (group.id=d00b2f341ffd4c80afe10b8079f151e3, role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=8a4d1bfa73f141cd90a91a1d9547a701) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:11.351667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce60ec-48ce-4470-a526-289852251aa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:11.353134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce60ec-48ce-4470-a526-289852251aa9 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:11.359614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 194/382] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 4 05:41:11 2022] GET /identity/v3/role_assignments?group.id=d00b2f341ffd4c80afe10b8079f151e3&scope.project.id=8a4d1bfa73f141cd90a91a1d9547a701&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:11.374921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9e7c2-f43b-4cfd-a7b0-31f23e8d2412 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:11.468728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:11.527529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9e7c2-f43b-4cfd-a7b0-31f23e8d2412 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:11.566865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710701d-f1a4-41de-87c8-ea02e9a4c377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:11.567710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710701d-f1a4-41de-87c8-ea02e9a4c377 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:11.567989 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710701d-f1a4-41de-87c8-ea02e9a4c377 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:11.568258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710701d-f1a4-41de-87c8-ea02e9a4c377 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:11.579616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710701d-f1a4-41de-87c8-ea02e9a4c377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-7710701d-f1a4-41de-87c8-ea02e9a4c377 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:11.587354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:11.588349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:11.589174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:11.589830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 189/383] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:11.596136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce60ec-48ce-4470-a526-289852251aa9 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:11.721273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:11.746309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce60ec-48ce-4470-a526-289852251aa9 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:11.792250 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0ce36-4524-433a-9c93-bcddd1a25c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:11.792932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ce36-4524-433a-9c93-bcddd1a25c2a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:11.793216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ce36-4524-433a-9c93-bcddd1a25c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:11.793490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ce36-4524-433a-9c93-bcddd1a25c2a None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:11.794252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0ce36-4524-433a-9c93-bcddd1a25c2a None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:11.794561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0ce36-4524-433a-9c93-bcddd1a25c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:11.795994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0ce36-4524-433a-9c93-bcddd1a25c2a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:11.807176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 195/384] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:11.812911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710701d-f1a4-41de-87c8-ea02e9a4c377 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:11.945981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710701d-f1a4-41de-87c8-ea02e9a4c377 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:11.973599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:11.987274 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bebcf8-89e9-4364-8520-dcc0d1a2868c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:11.988161 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bebcf8-89e9-4364-8520-dcc0d1a2868c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:11.988444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bebcf8-89e9-4364-8520-dcc0d1a2868c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:11.988710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bebcf8-89e9-4364-8520-dcc0d1a2868c None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:11.989882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bebcf8-89e9-4364-8520-dcc0d1a2868c None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:11.991340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bebcf8-89e9-4364-8520-dcc0d1a2868c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:11.993272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 190/385] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:11 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:11.999701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0ce36-4524-433a-9c93-bcddd1a25c2a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:12.140255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0ce36-4524-433a-9c93-bcddd1a25c2a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:12.172626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6aeef-d848-4973-8088-b3a4f13ff57c None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:12.172961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6aeef-d848-4973-8088-b3a4f13ff57c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:12.173176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6aeef-d848-4973-8088-b3a4f13ff57c None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:12.174832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 191/386] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:12.179975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df010eb-50b2-4217-a5cc-936f5b864ce0 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:12.180528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df010eb-50b2-4217-a5cc-936f5b864ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:12.181079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df010eb-50b2-4217-a5cc-936f5b864ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:12.185311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d137e-60fc-46bc-a08f-9409d0e5695c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:12.185311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d137e-60fc-46bc-a08f-9409d0e5695c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:12.185311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d137e-60fc-46bc-a08f-9409d0e5695c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:12.185311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d137e-60fc-46bc-a08f-9409d0e5695c None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:12.185781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d137e-60fc-46bc-a08f-9409d0e5695c None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:12.187225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d137e-60fc-46bc-a08f-9409d0e5695c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:12.194942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 196/387] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:11 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:12.196131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d137e-60fc-46bc-a08f-9409d0e5695c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:12.208175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2df010eb-50b2-4217-a5cc-936f5b864ce0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:12.209770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2df010eb-50b2-4217-a5cc-936f5b864ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:12.219581 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:12.350754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d137e-60fc-46bc-a08f-9409d0e5695c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:12.377003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 192/388] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:12.395950 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a3d8-08be-43c7-8f9c-3e08270cb5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:12.396862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a3d8-08be-43c7-8f9c-3e08270cb5dd None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:12.397145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a3d8-08be-43c7-8f9c-3e08270cb5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:12.397422 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a3d8-08be-43c7-8f9c-3e08270cb5dd None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04/groups/0b9ca3f758cc426988f62660078544b3/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:12.400905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a3d8-08be-43c7-8f9c-3e08270cb5dd None admin] RBAC: Authorizing `identity:create_grant(project_id=f35ff6000de749fc927ed896def15c04, group_id=0b9ca3f758cc426988f62660078544b3, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:12.402370 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a3d8-08be-43c7-8f9c-3e08270cb5dd None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:12.423182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df010eb-50b2-4217-a5cc-936f5b864ce0 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:12.435619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 197/389] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jul 4 05:41:12 2022] PUT /identity/v3/projects/f35ff6000de749fc927ed896def15c04/groups/0b9ca3f758cc426988f62660078544b3/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:12.553554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df010eb-50b2-4217-a5cc-936f5b864ce0 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:12.580402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:12.596240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a939fc1-feb6-4903-b6f3-a6553a755212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:12.597084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a939fc1-feb6-4903-b6f3-a6553a755212 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:12.597361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a939fc1-feb6-4903-b6f3-a6553a755212 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:12.597630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a939fc1-feb6-4903-b6f3-a6553a755212 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:12.609229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a939fc1-feb6-4903-b6f3-a6553a755212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-1a939fc1-feb6-4903-b6f3-a6553a755212 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:12.617982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:12.618976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:12.619843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 193/390] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:41:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:12.625829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a3d8-08be-43c7-8f9c-3e08270cb5dd None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:12.707689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:12.757831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a3d8-08be-43c7-8f9c-3e08270cb5dd None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:12.799996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbbc96-37f3-468e-956a-7eeb26ecaf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:12.800600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbbc96-37f3-468e-956a-7eeb26ecaf62 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:12.800883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbbc96-37f3-468e-956a-7eeb26ecaf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:12.801159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbbc96-37f3-468e-956a-7eeb26ecaf62 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:12.801921 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbbc96-37f3-468e-956a-7eeb26ecaf62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:12.802226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbbc96-37f3-468e-956a-7eeb26ecaf62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:12.803796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbbc96-37f3-468e-956a-7eeb26ecaf62 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:12.815274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 198/391] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:12.821380 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a939fc1-feb6-4903-b6f3-a6553a755212 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:12.948669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a939fc1-feb6-4903-b6f3-a6553a755212 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:12.984589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb7caf-bd0b-47dd-9409-a09215ebc8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:12.985466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb7caf-bd0b-47dd-9409-a09215ebc8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:12.985741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb7caf-bd0b-47dd-9409-a09215ebc8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:12.986009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb7caf-bd0b-47dd-9409-a09215ebc8c3 None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:12.987171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb7caf-bd0b-47dd-9409-a09215ebc8c3 None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:12.988703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb7caf-bd0b-47dd-9409-a09215ebc8c3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:12.990634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 194/392] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:12 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:12.996580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbbc96-37f3-468e-956a-7eeb26ecaf62 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:13.151090 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbbc96-37f3-468e-956a-7eeb26ecaf62 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:13.189232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:13.199826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7f042-f86d-4fe5-915f-cdba0c2bb8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:13.201005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7f042-f86d-4fe5-915f-cdba0c2bb8be None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:13.201481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7f042-f86d-4fe5-915f-cdba0c2bb8be None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:13.201954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7f042-f86d-4fe5-915f-cdba0c2bb8be None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:13.203514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7f042-f86d-4fe5-915f-cdba0c2bb8be None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:13.205168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7f042-f86d-4fe5-915f-cdba0c2bb8be None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:13.207288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 199/393] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:12 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:13.215280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb7caf-bd0b-47dd-9409-a09215ebc8c3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:13.296454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:13.320907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff554ae-5d81-434e-bd9f-fef53bbd476c None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:13.321522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff554ae-5d81-434e-bd9f-fef53bbd476c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:13.322071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff554ae-5d81-434e-bd9f-fef53bbd476c None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:13.324175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 200/394] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:41:13.329525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d15f-ec18-46ed-a8ce-02d8150d7c21 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:13.330129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d15f-ec18-46ed-a8ce-02d8150d7c21 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:13.330613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d15f-ec18-46ed-a8ce-02d8150d7c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:13.348605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-4e51d15f-ec18-46ed-a8ce-02d8150d7c21 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:13.350803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4e51d15f-ec18-46ed-a8ce-02d8150d7c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:13.373462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb7caf-bd0b-47dd-9409-a09215ebc8c3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:13.432037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 201/395] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:13.441812 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2afad-d3e1-4d0b-ae51-25d3306bbadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:13.442718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2afad-d3e1-4d0b-ae51-25d3306bbadb None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:13.442998 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2afad-d3e1-4d0b-ae51-25d3306bbadb None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:13.443296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2afad-d3e1-4d0b-ae51-25d3306bbadb None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04/groups/0b9ca3f758cc426988f62660078544b3/roles/d2054df5a6d24f5d90793b9b81cd4b4b` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:13.448055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2afad-d3e1-4d0b-ae51-25d3306bbadb None admin] RBAC: Authorizing `identity:create_grant(project_id=f35ff6000de749fc927ed896def15c04, group_id=0b9ca3f758cc426988f62660078544b3, role_id=d2054df5a6d24f5d90793b9b81cd4b4b)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:13.449527 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2afad-d3e1-4d0b-ae51-25d3306bbadb None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:13.456607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d15f-ec18-46ed-a8ce-02d8150d7c21 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:13.475869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 195/396] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jul 4 05:41:13 2022] PUT /identity/v3/projects/f35ff6000de749fc927ed896def15c04/groups/0b9ca3f758cc426988f62660078544b3/roles/d2054df5a6d24f5d90793b9b81cd4b4b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:13.577110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d15f-ec18-46ed-a8ce-02d8150d7c21 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:13.607808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:13.619281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ed104-e66d-40e1-97fd-bddf62917452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:13.620162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ed104-e66d-40e1-97fd-bddf62917452 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:13.620448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ed104-e66d-40e1-97fd-bddf62917452 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:13.620720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ed104-e66d-40e1-97fd-bddf62917452 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:13.639933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ed104-e66d-40e1-97fd-bddf62917452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-1c1ed104-e66d-40e1-97fd-bddf62917452 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:13.648177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:13.649152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:13.649946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 202/397] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:13.655735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2afad-d3e1-4d0b-ae51-25d3306bbadb None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:13.671078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234481-7be5-45ae-9869-93bd08c1865d None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:13.671440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234481-7be5-45ae-9869-93bd08c1865d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:13.671719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234481-7be5-45ae-9869-93bd08c1865d None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:13.673373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 203/398] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:13.678572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b018e-ffdf-49ea-b0d8-911573126ccc None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:13.678846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b018e-ffdf-49ea-b0d8-911573126ccc None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:13.679128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b018e-ffdf-49ea-b0d8-911573126ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:13.704358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-4c2b018e-ffdf-49ea-b0d8-911573126ccc None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:13.705763 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4c2b018e-ffdf-49ea-b0d8-911573126ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:13.789482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 204/399] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:13.800681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2afad-d3e1-4d0b-ae51-25d3306bbadb None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:13.811910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b018e-ffdf-49ea-b0d8-911573126ccc None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:13.812070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:13.844871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a7b0d-8277-4b9c-b96e-47a393d282fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:13.845859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a7b0d-8277-4b9c-b96e-47a393d282fe None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:13.846280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a7b0d-8277-4b9c-b96e-47a393d282fe None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:13.846675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a7b0d-8277-4b9c-b96e-47a393d282fe None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:13.847470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a7b0d-8277-4b9c-b96e-47a393d282fe None admin] RBAC: Adding query filter params (name=member) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:13.847777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a7b0d-8277-4b9c-b96e-47a393d282fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:13.850113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a7b0d-8277-4b9c-b96e-47a393d282fe None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:13.860554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 196/400] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:13.866090 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a7b0d-8277-4b9c-b96e-47a393d282fe None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:13.978191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b018e-ffdf-49ea-b0d8-911573126ccc None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:13.999109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:14.019636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c70163-aa48-4642-b131-facc0a8bb548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:14.020463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70163-aa48-4642-b131-facc0a8bb548 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.020742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70163-aa48-4642-b131-facc0a8bb548 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.021012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70163-aa48-4642-b131-facc0a8bb548 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.033312 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c70163-aa48-4642-b131-facc0a8bb548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-f2c70163-aa48-4642-b131-facc0a8bb548 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:14.042827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:14.043749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:14.044550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 205/401] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:41:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:14.050290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c70163-aa48-4642-b131-facc0a8bb548 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:14.116307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a7b0d-8277-4b9c-b96e-47a393d282fe None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:14.165476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cad0c-8d10-4634-8d73-b839eb18f5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:14.167118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cad0c-8d10-4634-8d73-b839eb18f5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.167752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cad0c-8d10-4634-8d73-b839eb18f5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.168033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cad0c-8d10-4634-8d73-b839eb18f5f1 None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.169644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cad0c-8d10-4634-8d73-b839eb18f5f1 None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:14.172040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cad0c-8d10-4634-8d73-b839eb18f5f1 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:14.174750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 197/402] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:13 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:14.176086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b64c-5c3e-48a3-8d41-b03bc7ce165c None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.176357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b64c-5c3e-48a3-8d41-b03bc7ce165c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.176633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b64c-5c3e-48a3-8d41-b03bc7ce165c None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.182787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 198/403] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:14.183806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06b64c-5c3e-48a3-8d41-b03bc7ce165c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:14.196066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c70163-aa48-4642-b131-facc0a8bb548 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:14.233326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb53e00-2b26-4b27-b47a-8ff09d1ffcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:14.233948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb53e00-2b26-4b27-b47a-8ff09d1ffcc3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.234237 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb53e00-2b26-4b27-b47a-8ff09d1ffcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.234511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb53e00-2b26-4b27-b47a-8ff09d1ffcc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.235279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb53e00-2b26-4b27-b47a-8ff09d1ffcc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:14.235584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb53e00-2b26-4b27-b47a-8ff09d1ffcc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:14.236995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb53e00-2b26-4b27-b47a-8ff09d1ffcc3 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:14.240642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:14.246337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 206/404] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:14.247776 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecd36f-b4f0-4fb1-b56f-58e57216781e None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.248068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecd36f-b4f0-4fb1-b56f-58e57216781e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.248345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecd36f-b4f0-4fb1-b56f-58e57216781e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.267531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d7ecd36f-b4f0-4fb1-b56f-58e57216781e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:14.268716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d7ecd36f-b4f0-4fb1-b56f-58e57216781e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:14.305630 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:14.322092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06b64c-5c3e-48a3-8d41-b03bc7ce165c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:14.356803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47de1f-f204-47ff-9f51-1b9b18454e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:14.357678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47de1f-f204-47ff-9f51-1b9b18454e52 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.357956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47de1f-f204-47ff-9f51-1b9b18454e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.358226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47de1f-f204-47ff-9f51-1b9b18454e52 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.359331 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 207/405] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:14.360195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecd36f-b4f0-4fb1-b56f-58e57216781e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:14.368986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47de1f-f204-47ff-9f51-1b9b18454e52 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:14.370431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47de1f-f204-47ff-9f51-1b9b18454e52 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:14.372304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 199/406] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:14 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:14.378220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47de1f-f204-47ff-9f51-1b9b18454e52 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:14.483864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecd36f-b4f0-4fb1-b56f-58e57216781e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:14.512699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47de1f-f204-47ff-9f51-1b9b18454e52 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:14.523669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:14.532459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65573a-4abb-4576-b044-ec5416d68cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:14.533823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65573a-4abb-4576-b044-ec5416d68cbd None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.534321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65573a-4abb-4576-b044-ec5416d68cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.534788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65573a-4abb-4576-b044-ec5416d68cbd None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.536230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65573a-4abb-4576-b044-ec5416d68cbd None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:14.537953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65573a-4abb-4576-b044-ec5416d68cbd None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:14.540549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 208/407] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:14 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:14.541586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65573a-4abb-4576-b044-ec5416d68cbd None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:14.555589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a758a-fdc6-41e7-942e-0042365eb95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:14.556889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a758a-fdc6-41e7-942e-0042365eb95c None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.557501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a758a-fdc6-41e7-942e-0042365eb95c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.557966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a758a-fdc6-41e7-942e-0042365eb95c None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74/groups/0b9ca3f758cc426988f62660078544b3/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.560432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a758a-fdc6-41e7-942e-0042365eb95c None admin] RBAC: Authorizing `identity:create_grant(project_id=81bfce5971f14bb58ff604d83072bd74, group_id=0b9ca3f758cc426988f62660078544b3, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:14.562283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a758a-fdc6-41e7-942e-0042365eb95c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:14.586736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 200/408] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jul 4 05:41:14 2022] PUT /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74/groups/0b9ca3f758cc426988f62660078544b3/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:14.588628 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a758a-fdc6-41e7-942e-0042365eb95c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:14.590666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:14.656164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65573a-4abb-4576-b044-ec5416d68cbd None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:14.692253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f820d51-4cab-4418-8d10-6fbb3fa3cfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:14.692920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f820d51-4cab-4418-8d10-6fbb3fa3cfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.693419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f820d51-4cab-4418-8d10-6fbb3fa3cfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.693689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f820d51-4cab-4418-8d10-6fbb3fa3cfb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.704832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f820d51-4cab-4418-8d10-6fbb3fa3cfb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-2f820d51-4cab-4418-8d10-6fbb3fa3cfb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:14.715309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:14.716247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:14.717026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 209/409] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:14.727692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f820d51-4cab-4418-8d10-6fbb3fa3cfb7 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:14.798604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a758a-fdc6-41e7-942e-0042365eb95c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:14.868017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106fe28-6729-4ad5-98b5-08fb6719688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:14.869653 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106fe28-6729-4ad5-98b5-08fb6719688c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.870082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106fe28-6729-4ad5-98b5-08fb6719688c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.870489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106fe28-6729-4ad5-98b5-08fb6719688c None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.872532 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106fe28-6729-4ad5-98b5-08fb6719688c None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:14.874126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106fe28-6729-4ad5-98b5-08fb6719688c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:14.876993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 201/410] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:14 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:14.883212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106fe28-6729-4ad5-98b5-08fb6719688c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:14.896037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:14.904739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f820d51-4cab-4418-8d10-6fbb3fa3cfb7 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:14.939026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:14.946352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66847b-cc27-45e0-935c-6eab151f2550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:14.947641 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66847b-cc27-45e0-935c-6eab151f2550 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:14.948139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66847b-cc27-45e0-935c-6eab151f2550 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:14.948755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66847b-cc27-45e0-935c-6eab151f2550 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:14.949831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66847b-cc27-45e0-935c-6eab151f2550 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:14.950324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66847b-cc27-45e0-935c-6eab151f2550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:14.951975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66847b-cc27-45e0-935c-6eab151f2550 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:14.961711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 210/411] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:14.968698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66847b-cc27-45e0-935c-6eab151f2550 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:15.059377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106fe28-6729-4ad5-98b5-08fb6719688c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:15.084422 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66847b-cc27-45e0-935c-6eab151f2550 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:15.120192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a5f3f-8fb9-4c4c-860b-ba463c68988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:15.121095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a5f3f-8fb9-4c4c-860b-ba463c68988a None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:15.121417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a5f3f-8fb9-4c4c-860b-ba463c68988a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:15.121642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a5f3f-8fb9-4c4c-860b-ba463c68988a None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74/groups/0b9ca3f758cc426988f62660078544b3/roles/d2054df5a6d24f5d90793b9b81cd4b4b` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:15.123940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a5f3f-8fb9-4c4c-860b-ba463c68988a None admin] RBAC: Authorizing `identity:create_grant(project_id=81bfce5971f14bb58ff604d83072bd74, group_id=0b9ca3f758cc426988f62660078544b3, role_id=d2054df5a6d24f5d90793b9b81cd4b4b)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:15.125318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6393021-bfe7-4024-ac66-f41bc2437615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:15.125437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a5f3f-8fb9-4c4c-860b-ba463c68988a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:15.126506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6393021-bfe7-4024-ac66-f41bc2437615 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:15.126989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6393021-bfe7-4024-ac66-f41bc2437615 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:15.127482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6393021-bfe7-4024-ac66-f41bc2437615 None admin] PATH_INFO: `/v3/groups/d00b2f341ffd4c80afe10b8079f151e3` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:15.129514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6393021-bfe7-4024-ac66-f41bc2437615 None admin] RBAC: Authorizing `identity:get_group(group_id=d00b2f341ffd4c80afe10b8079f151e3)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:15.131134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6393021-bfe7-4024-ac66-f41bc2437615 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:15.137611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 211/412] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:14 2022] GET /identity/v3/groups/d00b2f341ffd4c80afe10b8079f151e3 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:15.137611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4f5ec-46e2-4e9d-8f2e-e67c03c4fa21 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:15.137611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4f5ec-46e2-4e9d-8f2e-e67c03c4fa21 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:15.137611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4f5ec-46e2-4e9d-8f2e-e67c03c4fa21 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:15.137611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 212/413] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:15.141214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4f5ec-46e2-4e9d-8f2e-e67c03c4fa21 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:15.155425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 202/414] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jul 4 05:41:14 2022] PUT /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74/groups/0b9ca3f758cc426988f62660078544b3/roles/d2054df5a6d24f5d90793b9b81cd4b4b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:15.157322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005be8-da17-4612-97bd-092b7eea7d6c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:15.157904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005be8-da17-4612-97bd-092b7eea7d6c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:15.158373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005be8-da17-4612-97bd-092b7eea7d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:15.175756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:15.192540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ed005be8-da17-4612-97bd-092b7eea7d6c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:15.194558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ed005be8-da17-4612-97bd-092b7eea7d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:15.265514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4f5ec-46e2-4e9d-8f2e-e67c03c4fa21 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:15.306138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aba15-b171-4a69-b88c-0a6710486cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:15.307382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aba15-b171-4a69-b88c-0a6710486cfc None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:15.307751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aba15-b171-4a69-b88c-0a6710486cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:15.308021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aba15-b171-4a69-b88c-0a6710486cfc None admin] PATH_INFO: `/v3/projects/8a4d1bfa73f141cd90a91a1d9547a701` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:15.309571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aba15-b171-4a69-b88c-0a6710486cfc None admin] RBAC: Authorizing `identity:get_project(project_id=8a4d1bfa73f141cd90a91a1d9547a701)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:15.311648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aba15-b171-4a69-b88c-0a6710486cfc None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:15.313974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 213/415] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:15 2022] GET /identity/v3/projects/8a4d1bfa73f141cd90a91a1d9547a701 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:15.323227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aba15-b171-4a69-b88c-0a6710486cfc None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:15.347874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 203/416] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:15.369844 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005be8-da17-4612-97bd-092b7eea7d6c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:15.435283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:15.470999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:15.481025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aba15-b171-4a69-b88c-0a6710486cfc None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:15.527692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005be8-da17-4612-97bd-092b7eea7d6c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:15.528692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55590668-fe75-4955-84b5-2d539300f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:15.529940 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55590668-fe75-4955-84b5-2d539300f628 None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:15.530426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55590668-fe75-4955-84b5-2d539300f628 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:15.530895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55590668-fe75-4955-84b5-2d539300f628 None admin] PATH_INFO: `/v3/projects/8a4d1bfa73f141cd90a91a1d9547a701/groups/d00b2f341ffd4c80afe10b8079f151e3/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:15.533155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55590668-fe75-4955-84b5-2d539300f628 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4d1bfa73f141cd90a91a1d9547a701, group_id=d00b2f341ffd4c80afe10b8079f151e3, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:15.534943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55590668-fe75-4955-84b5-2d539300f628 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:15.563503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 214/417] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jul 4 05:41:15 2022] PUT /identity/v3/projects/8a4d1bfa73f141cd90a91a1d9547a701/groups/d00b2f341ffd4c80afe10b8079f151e3/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:15.587097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e49c6-d7e5-44f3-ae34-8f1acc4b048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:15.588571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e49c6-d7e5-44f3-ae34-8f1acc4b048a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:15.589196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e49c6-d7e5-44f3-ae34-8f1acc4b048a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:15.589657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e49c6-d7e5-44f3-ae34-8f1acc4b048a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:15.604943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e49c6-d7e5-44f3-ae34-8f1acc4b048a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-f00e49c6-d7e5-44f3-ae34-8f1acc4b048a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:15.613383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:15.614278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:15.615080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:15.621655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 204/418] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:15.628661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55590668-fe75-4955-84b5-2d539300f628 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:15.856694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55590668-fe75-4955-84b5-2d539300f628 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:15.915759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b99e93-bba4-463a-8ba5-e49da1369bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:15.917064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b99e93-bba4-463a-8ba5-e49da1369bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:15.917757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b99e93-bba4-463a-8ba5-e49da1369bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:15.920105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b99e93-bba4-463a-8ba5-e49da1369bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:15.921472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b99e93-bba4-463a-8ba5-e49da1369bb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:15.923610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b99e93-bba4-463a-8ba5-e49da1369bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:15.928464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b99e93-bba4-463a-8ba5-e49da1369bb7 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:15.946888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 215/419] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:15.952545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e49c6-d7e5-44f3-ae34-8f1acc4b048a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:15.986006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4a368-0e67-4749-8e2f-71c22617ffca None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:15.986358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4a368-0e67-4749-8e2f-71c22617ffca None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:15.986564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4a368-0e67-4749-8e2f-71c22617ffca None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:15.988004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 216/420] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:15.992828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b08513-88cc-478f-b7fd-b587af72c2ca None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:15.993116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b08513-88cc-478f-b7fd-b587af72c2ca None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:15.993399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b08513-88cc-478f-b7fd-b587af72c2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:16.023452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b1b08513-88cc-478f-b7fd-b587af72c2ca None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:16.024675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b1b08513-88cc-478f-b7fd-b587af72c2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:16.049167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:16.067033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:16.086927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e49c6-d7e5-44f3-ae34-8f1acc4b048a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:16.135848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ff7f8-2a8e-4d52-87ab-37e00e6a8bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:16.136901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ff7f8-2a8e-4d52-87ab-37e00e6a8bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:16.137395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ff7f8-2a8e-4d52-87ab-37e00e6a8bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:16.137870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ff7f8-2a8e-4d52-87ab-37e00e6a8bd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:16.150256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 217/421] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:16.181794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b08513-88cc-478f-b7fd-b587af72c2ca None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:16.219818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ff7f8-2a8e-4d52-87ab-37e00e6a8bd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:16.221310 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ff7f8-2a8e-4d52-87ab-37e00e6a8bd7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:16.323311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b08513-88cc-478f-b7fd-b587af72c2ca None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:16.355014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:16.379071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 205/422] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:16.386934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ff7f8-2a8e-4d52-87ab-37e00e6a8bd7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:16.391110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77a2eb-5f9e-404e-a3ab-b29c80dcaf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:16.392028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77a2eb-5f9e-404e-a3ab-b29c80dcaf8f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:16.392305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77a2eb-5f9e-404e-a3ab-b29c80dcaf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:16.392592 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77a2eb-5f9e-404e-a3ab-b29c80dcaf8f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:16.413433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77a2eb-5f9e-404e-a3ab-b29c80dcaf8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-4b77a2eb-5f9e-404e-a3ab-b29c80dcaf8f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:16.421942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:16.422804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:16.423618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:16.424222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 218/423] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:41:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:16.436163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77a2eb-5f9e-404e-a3ab-b29c80dcaf8f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:16.452608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:16.516707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ff7f8-2a8e-4d52-87ab-37e00e6a8bd7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:16.555536 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085fd2e-5b10-442f-8b51-7d251383a5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:16.557571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085fd2e-5b10-442f-8b51-7d251383a5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:16.558145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085fd2e-5b10-442f-8b51-7d251383a5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:16.559668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085fd2e-5b10-442f-8b51-7d251383a5a2 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:16.561782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085fd2e-5b10-442f-8b51-7d251383a5a2 None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:16.563605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085fd2e-5b10-442f-8b51-7d251383a5a2 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:16.568113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 206/424] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:16 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:16.576215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085fd2e-5b10-442f-8b51-7d251383a5a2 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:16.662065 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:16.663288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77a2eb-5f9e-404e-a3ab-b29c80dcaf8f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:16.701323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b1376-db7f-4174-a50e-c44bd4820072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:16.702020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b1376-db7f-4174-a50e-c44bd4820072 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:16.702298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b1376-db7f-4174-a50e-c44bd4820072 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:16.702570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b1376-db7f-4174-a50e-c44bd4820072 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:16.703398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b1376-db7f-4174-a50e-c44bd4820072 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:16.703698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b1376-db7f-4174-a50e-c44bd4820072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:16.706869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b1376-db7f-4174-a50e-c44bd4820072 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:16.715494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 219/425] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:16.728425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b1376-db7f-4174-a50e-c44bd4820072 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:16.739205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085fd2e-5b10-442f-8b51-7d251383a5a2 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:16.802458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd0dbd-9c72-4124-859c-0bc8bc226503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:16.803735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd0dbd-9c72-4124-859c-0bc8bc226503 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:16.804206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd0dbd-9c72-4124-859c-0bc8bc226503 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:16.804663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd0dbd-9c72-4124-859c-0bc8bc226503 None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:16.808098 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd0dbd-9c72-4124-859c-0bc8bc226503 None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:16.809792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd0dbd-9c72-4124-859c-0bc8bc226503 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:16.812763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 207/426] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:16 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:16.819984 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd0dbd-9c72-4124-859c-0bc8bc226503 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:16.864320 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:16.934241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b1376-db7f-4174-a50e-c44bd4820072 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:17.008577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead30a42-0d3d-4d55-9671-8f8f79c9eacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:17.009706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead30a42-0d3d-4d55-9671-8f8f79c9eacc None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.010357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead30a42-0d3d-4d55-9671-8f8f79c9eacc None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.010876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead30a42-0d3d-4d55-9671-8f8f79c9eacc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.017723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd0dbd-9c72-4124-859c-0bc8bc226503 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:17.063506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:17.090188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11deac-f6d2-43ba-8016-7320bf5bcef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:17.091217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11deac-f6d2-43ba-8016-7320bf5bcef8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.091720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11deac-f6d2-43ba-8016-7320bf5bcef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.092176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11deac-f6d2-43ba-8016-7320bf5bcef8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.093213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11deac-f6d2-43ba-8016-7320bf5bcef8 None admin] RBAC: Adding query filter params (group.id=0b9ca3f758cc426988f62660078544b3, role.id=d0a32856510a480a897677657ace20de, scope.project.id=f35ff6000de749fc927ed896def15c04) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:17.093707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11deac-f6d2-43ba-8016-7320bf5bcef8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:17.095359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11deac-f6d2-43ba-8016-7320bf5bcef8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:17.105151 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 208/427] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 4 05:41:16 2022] GET /identity/v3/role_assignments?group.id=0b9ca3f758cc426988f62660078544b3&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=d0a32856510a480a897677657ace20de => generated 611 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:17.121217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a892dd-0683-43c6-8311-5d72881b44c3 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.121931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a892dd-0683-43c6-8311-5d72881b44c3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.122392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a892dd-0683-43c6-8311-5d72881b44c3 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.124411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 209/428] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:41:17.124830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead30a42-0d3d-4d55-9671-8f8f79c9eacc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:17.133056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead30a42-0d3d-4d55-9671-8f8f79c9eacc None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:17.133269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae700a-b83e-4dd8-9009-2b8c8483349b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.133919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae700a-b83e-4dd8-9009-2b8c8483349b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.134384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae700a-b83e-4dd8-9009-2b8c8483349b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.164602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4aae700a-b83e-4dd8-9009-2b8c8483349b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:17.167186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4aae700a-b83e-4dd8-9009-2b8c8483349b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:17.235102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:17.267799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 210/429] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:17.294865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 220/430] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:17.301008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae700a-b83e-4dd8-9009-2b8c8483349b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:17.333189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead30a42-0d3d-4d55-9671-8f8f79c9eacc None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:17.443152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae700a-b83e-4dd8-9009-2b8c8483349b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:17.462629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:17.464729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead30a42-0d3d-4d55-9671-8f8f79c9eacc None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:17.487467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186d590-8023-40f5-a6f8-b78d29790fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:17.488728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186d590-8023-40f5-a6f8-b78d29790fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.491090 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186d590-8023-40f5-a6f8-b78d29790fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.491655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186d590-8023-40f5-a6f8-b78d29790fa8 None admin] PATH_INFO: `/v3/projects/f35ff6000de749fc927ed896def15c04` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.493297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186d590-8023-40f5-a6f8-b78d29790fa8 None admin] RBAC: Authorizing `identity:get_project(project_id=f35ff6000de749fc927ed896def15c04)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:17.495132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186d590-8023-40f5-a6f8-b78d29790fa8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:17.497434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 211/431] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:17 2022] GET /identity/v3/projects/f35ff6000de749fc927ed896def15c04 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:17.503055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186d590-8023-40f5-a6f8-b78d29790fa8 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:17.507681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af7cf8-2f01-4176-a3ab-c2a78fce0fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:17.509475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af7cf8-2f01-4176-a3ab-c2a78fce0fed None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.509785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af7cf8-2f01-4176-a3ab-c2a78fce0fed None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.510067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af7cf8-2f01-4176-a3ab-c2a78fce0fed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.523936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af7cf8-2f01-4176-a3ab-c2a78fce0fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-79af7cf8-2f01-4176-a3ab-c2a78fce0fed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:17.534369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:17.535336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:17.536183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 221/432] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:17.541152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af7cf8-2f01-4176-a3ab-c2a78fce0fed None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:17.576934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:17.624540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186d590-8023-40f5-a6f8-b78d29790fa8 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:17.659354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af7cf8-2f01-4176-a3ab-c2a78fce0fed None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:17.659945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea2814-94f9-4831-9982-3ab23d1fdf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:17.661143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea2814-94f9-4831-9982-3ab23d1fdf86 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.661620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea2814-94f9-4831-9982-3ab23d1fdf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.662099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea2814-94f9-4831-9982-3ab23d1fdf86 None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.663586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea2814-94f9-4831-9982-3ab23d1fdf86 None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:17.665261 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea2814-94f9-4831-9982-3ab23d1fdf86 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:17.667583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 212/433] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:17 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:17.673495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea2814-94f9-4831-9982-3ab23d1fdf86 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:17.696365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6af6-45d1-4c44-9d9b-068c7c2be1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:17.697090 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6af6-45d1-4c44-9d9b-068c7c2be1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.697379 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6af6-45d1-4c44-9d9b-068c7c2be1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.697651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6af6-45d1-4c44-9d9b-068c7c2be1f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.698526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc6af6-45d1-4c44-9d9b-068c7c2be1f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:17.698828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc6af6-45d1-4c44-9d9b-068c7c2be1f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:17.700273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc6af6-45d1-4c44-9d9b-068c7c2be1f4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:17.710204 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 222/434] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:41:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:17.715764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6af6-45d1-4c44-9d9b-068c7c2be1f4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:17.731982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:17.782752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea2814-94f9-4831-9982-3ab23d1fdf86 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:17.823350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:17.835220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6af6-45d1-4c44-9d9b-068c7c2be1f4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:17.838718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2dbc0-14b3-482c-a3b9-c704680ac344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:17.839227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2dbc0-14b3-482c-a3b9-c704680ac344 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.839531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2dbc0-14b3-482c-a3b9-c704680ac344 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.839806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2dbc0-14b3-482c-a3b9-c704680ac344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.841832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2dbc0-14b3-482c-a3b9-c704680ac344 None admin] RBAC: Adding query filter params (group.id=0b9ca3f758cc426988f62660078544b3, role.id=d2054df5a6d24f5d90793b9b81cd4b4b, scope.project.id=f35ff6000de749fc927ed896def15c04) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:17.841832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2dbc0-14b3-482c-a3b9-c704680ac344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:17.842421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2dbc0-14b3-482c-a3b9-c704680ac344 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:17.860227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 213/435] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 4 05:41:17 2022] GET /identity/v3/role_assignments?group.id=0b9ca3f758cc426988f62660078544b3&scope.project.id=f35ff6000de749fc927ed896def15c04&role.id=d2054df5a6d24f5d90793b9b81cd4b4b => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:17.901555 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a3478-9068-4c7b-8867-584f10fb3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:17.902542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a3478-9068-4c7b-8867-584f10fb3657 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.902965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a3478-9068-4c7b-8867-584f10fb3657 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.903416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a3478-9068-4c7b-8867-584f10fb3657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.938676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2ce61-b79b-4e9c-8978-9f806513f962 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.939231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2ce61-b79b-4e9c-8978-9f806513f962 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.939725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2ce61-b79b-4e9c-8978-9f806513f962 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.941368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 214/436] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:41:17.948397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791f5a-3ecb-4c3a-a786-ac38dd047a8f None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:17.948681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791f5a-3ecb-4c3a-a786-ac38dd047a8f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:17.948964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791f5a-3ecb-4c3a-a786-ac38dd047a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:17.977556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-fd791f5a-3ecb-4c3a-a786-ac38dd047a8f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:17.977556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fd791f5a-3ecb-4c3a-a786-ac38dd047a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:18.008479 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a3478-9068-4c7b-8867-584f10fb3657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:18.010102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a3478-9068-4c7b-8867-584f10fb3657 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:18.063582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 215/437] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:18.064883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e034c-9239-40db-811c-c9ac6010b98e None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.065170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e034c-9239-40db-811c-c9ac6010b98e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.065445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e034c-9239-40db-811c-c9ac6010b98e None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.071044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 216/438] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:18.072980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed393f-76df-4186-835c-78de6edf7d1d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.073364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed393f-76df-4186-835c-78de6edf7d1d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.073739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed393f-76df-4186-835c-78de6edf7d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.094948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c0ed393f-76df-4186-835c-78de6edf7d1d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:18.097194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c0ed393f-76df-4186-835c-78de6edf7d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:18.121914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:18.135292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 223/439] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 05:41:18.135817 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a3478-9068-4c7b-8867-584f10fb3657 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:18.150785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:18.182578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 217/440] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:18.183766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed393f-76df-4186-835c-78de6edf7d1d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:18.256896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a3478-9068-4c7b-8867-584f10fb3657 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:18.292351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed393f-76df-4186-835c-78de6edf7d1d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:18.296364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363ac04-fe5b-463c-b858-4bbd2b1a5c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:18.297242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ac04-fe5b-463c-b858-4bbd2b1a5c92 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.297518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ac04-fe5b-463c-b858-4bbd2b1a5c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.297786 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ac04-fe5b-463c-b858-4bbd2b1a5c92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.308200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363ac04-fe5b-463c-b858-4bbd2b1a5c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-d363ac04-fe5b-463c-b858-4bbd2b1a5c92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:18.316408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:18.317277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:18.318088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 224/441] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:41:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:18.318756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363ac04-fe5b-463c-b858-4bbd2b1a5c92 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:18.330777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f23fd-34d0-4e40-a153-cdc2f1ab03c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:18.331683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f23fd-34d0-4e40-a153-cdc2f1ab03c4 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.331967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f23fd-34d0-4e40-a153-cdc2f1ab03c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.332237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f23fd-34d0-4e40-a153-cdc2f1ab03c4 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.333406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f23fd-34d0-4e40-a153-cdc2f1ab03c4 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:18.334856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f23fd-34d0-4e40-a153-cdc2f1ab03c4 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:18.336710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 218/442] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:18 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:18.337389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f23fd-34d0-4e40-a153-cdc2f1ab03c4 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:18.369591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:18.380163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:18.422371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363ac04-fe5b-463c-b858-4bbd2b1a5c92 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:18.436512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f23fd-34d0-4e40-a153-cdc2f1ab03c4 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:18.459109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ef542-1dd5-43a3-af00-81987df55b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:18.459931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef542-1dd5-43a3-af00-81987df55b5e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.460206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef542-1dd5-43a3-af00-81987df55b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.460461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef542-1dd5-43a3-af00-81987df55b5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.470618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ef542-1dd5-43a3-af00-81987df55b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:18.471500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9315138-3ae1-44f0-8d09-7a827175ffca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:18.472177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9315138-3ae1-44f0-8d09-7a827175ffca None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.472464 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9315138-3ae1-44f0-8d09-7a827175ffca None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.472736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9315138-3ae1-44f0-8d09-7a827175ffca None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.473494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9315138-3ae1-44f0-8d09-7a827175ffca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:18.473801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9315138-3ae1-44f0-8d09-7a827175ffca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:18.475182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9315138-3ae1-44f0-8d09-7a827175ffca None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-812ef542-1dd5-43a3-af00-81987df55b5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:18.478210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:18.479092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:18.479949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 225/443] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:18.480598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ef542-1dd5-43a3-af00-81987df55b5e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:18.492006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 219/444] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:18.492006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9315138-3ae1-44f0-8d09-7a827175ffca None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:18.585239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ef542-1dd5-43a3-af00-81987df55b5e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:18.588434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9315138-3ae1-44f0-8d09-7a827175ffca None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:18.595180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:18.595394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:18.619330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1dc55-416e-4260-9938-783007ca3308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:18.620191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1dc55-416e-4260-9938-783007ca3308 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.620468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1dc55-416e-4260-9938-783007ca3308 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.620735 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1dc55-416e-4260-9938-783007ca3308 None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.621913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1dc55-416e-4260-9938-783007ca3308 None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:18.623426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1dc55-416e-4260-9938-783007ca3308 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:18.623829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f44c5-ebec-4afb-8f9c-5926563f76d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:18.624507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f44c5-ebec-4afb-8f9c-5926563f76d7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.624792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f44c5-ebec-4afb-8f9c-5926563f76d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.625056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 226/445] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:18 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:18.625170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f44c5-ebec-4afb-8f9c-5926563f76d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.625809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f44c5-ebec-4afb-8f9c-5926563f76d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:18.625902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1dc55-416e-4260-9938-783007ca3308 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:18.626128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f44c5-ebec-4afb-8f9c-5926563f76d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:18.627505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f44c5-ebec-4afb-8f9c-5926563f76d7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:18.636092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 220/446] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:18.636891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f44c5-ebec-4afb-8f9c-5926563f76d7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:18.722687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1dc55-416e-4260-9938-783007ca3308 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:18.733757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f44c5-ebec-4afb-8f9c-5926563f76d7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:18.754672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d149a-e833-4ad7-b694-ed75e890012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:18.755342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d149a-e833-4ad7-b694-ed75e890012f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.755642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d149a-e833-4ad7-b694-ed75e890012f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.755895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d149a-e833-4ad7-b694-ed75e890012f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.766753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ae865-855d-4ab7-b2fc-18b207962ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:18.767443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ae865-855d-4ab7-b2fc-18b207962ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.767728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ae865-855d-4ab7-b2fc-18b207962ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.768006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ae865-855d-4ab7-b2fc-18b207962ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.768840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ae865-855d-4ab7-b2fc-18b207962ff0 None admin] RBAC: Adding query filter params (group.id=0b9ca3f758cc426988f62660078544b3, role.id=d0a32856510a480a897677657ace20de, scope.project.id=81bfce5971f14bb58ff604d83072bd74) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:18.769150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ae865-855d-4ab7-b2fc-18b207962ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:18.770583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ae865-855d-4ab7-b2fc-18b207962ff0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:18.776482 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 221/447] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 4 05:41:18 2022] GET /identity/v3/role_assignments?group.id=0b9ca3f758cc426988f62660078544b3&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=d0a32856510a480a897677657ace20de => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:18.777275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ae865-855d-4ab7-b2fc-18b207962ff0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:18.803952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:18.825498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:18.834420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d149a-e833-4ad7-b694-ed75e890012f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:18.836524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d149a-e833-4ad7-b694-ed75e890012f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:18.913311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ae865-855d-4ab7-b2fc-18b207962ff0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:18.952644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f1e2d-d7f5-4a25-bc8d-620be14cfccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:18.953315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f1e2d-d7f5-4a25-bc8d-620be14cfccd None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:18.953585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f1e2d-d7f5-4a25-bc8d-620be14cfccd None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:18.953855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f1e2d-d7f5-4a25-bc8d-620be14cfccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:18.964219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 227/448] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:18.970031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d149a-e833-4ad7-b694-ed75e890012f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:19.035842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f1e2d-d7f5-4a25-bc8d-620be14cfccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:19.037332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f1e2d-d7f5-4a25-bc8d-620be14cfccd None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:19.063714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:19.074777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:19.093099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d149a-e833-4ad7-b694-ed75e890012f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:19.130629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c475c6-25a3-4964-889a-7170447cf150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:19.131597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c475c6-25a3-4964-889a-7170447cf150 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:19.131878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c475c6-25a3-4964-889a-7170447cf150 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:19.132148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c475c6-25a3-4964-889a-7170447cf150 None admin] PATH_INFO: `/v3/projects/81bfce5971f14bb58ff604d83072bd74` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:19.133343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c475c6-25a3-4964-889a-7170447cf150 None admin] RBAC: Authorizing `identity:get_project(project_id=81bfce5971f14bb58ff604d83072bd74)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:19.134848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c475c6-25a3-4964-889a-7170447cf150 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:19.136788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 228/449] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:18 2022] GET /identity/v3/projects/81bfce5971f14bb58ff604d83072bd74 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:19.142277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c475c6-25a3-4964-889a-7170447cf150 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:19.154208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 222/450] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:19.159553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f1e2d-d7f5-4a25-bc8d-620be14cfccd None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:19.241504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c475c6-25a3-4964-889a-7170447cf150 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:19.257699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f1e2d-d7f5-4a25-bc8d-620be14cfccd None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:19.275215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b8e4e-2e63-4f88-a2e6-3e1345827427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:19.276115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b8e4e-2e63-4f88-a2e6-3e1345827427 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:19.276396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b8e4e-2e63-4f88-a2e6-3e1345827427 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:19.276668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b8e4e-2e63-4f88-a2e6-3e1345827427 None admin] PATH_INFO: `/v3/groups/0b9ca3f758cc426988f62660078544b3` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:19.277838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b8e4e-2e63-4f88-a2e6-3e1345827427 None admin] RBAC: Authorizing `identity:get_group(group_id=0b9ca3f758cc426988f62660078544b3)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:19.279321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b8e4e-2e63-4f88-a2e6-3e1345827427 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:19.280985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 229/451] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:19 2022] GET /identity/v3/groups/0b9ca3f758cc426988f62660078544b3 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:19.286112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:19.286997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b8e4e-2e63-4f88-a2e6-3e1345827427 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:19.291467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc360c-a188-4535-9e0b-5cbd271096e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:19.292336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc360c-a188-4535-9e0b-5cbd271096e4 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:19.292616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc360c-a188-4535-9e0b-5cbd271096e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:19.292889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc360c-a188-4535-9e0b-5cbd271096e4 None admin] PATH_INFO: `/v3/projects/8a4d1bfa73f141cd90a91a1d9547a701` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:19.294055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc360c-a188-4535-9e0b-5cbd271096e4 None admin] RBAC: Authorizing `identity:get_project(project_id=8a4d1bfa73f141cd90a91a1d9547a701)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:19.295610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc360c-a188-4535-9e0b-5cbd271096e4 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:19.297311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 223/452] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:41:19 2022] GET /identity/v3/projects/8a4d1bfa73f141cd90a91a1d9547a701 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:19.302714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc360c-a188-4535-9e0b-5cbd271096e4 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:19.303477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:19.385643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b8e4e-2e63-4f88-a2e6-3e1345827427 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:19.403112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc360c-a188-4535-9e0b-5cbd271096e4 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:19.419709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e9587-bb53-45f1-90cd-2a697e1146ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:19.420177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e9587-bb53-45f1-90cd-2a697e1146ae None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:19.420456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e9587-bb53-45f1-90cd-2a697e1146ae None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:19.420725 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e9587-bb53-45f1-90cd-2a697e1146ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:19.421553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e9587-bb53-45f1-90cd-2a697e1146ae None admin] RBAC: Adding query filter params (group.id=0b9ca3f758cc426988f62660078544b3, role.id=d2054df5a6d24f5d90793b9b81cd4b4b, scope.project.id=81bfce5971f14bb58ff604d83072bd74) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:19.421893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e9587-bb53-45f1-90cd-2a697e1146ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:19.423352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e9587-bb53-45f1-90cd-2a697e1146ae None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:19.429321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 230/453] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 4 05:41:19 2022] GET /identity/v3/role_assignments?group.id=0b9ca3f758cc426988f62660078544b3&scope.project.id=81bfce5971f14bb58ff604d83072bd74&role.id=d2054df5a6d24f5d90793b9b81cd4b4b => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:19.436222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2877ca8-4109-449e-9dcf-e2e21320b3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:19.437111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2877ca8-4109-449e-9dcf-e2e21320b3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:19.437391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2877ca8-4109-449e-9dcf-e2e21320b3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:19.437662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2877ca8-4109-449e-9dcf-e2e21320b3f6 None admin] PATH_INFO: `/v3/groups/d00b2f341ffd4c80afe10b8079f151e3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:19.438926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2877ca8-4109-449e-9dcf-e2e21320b3f6 None admin] RBAC: Authorizing `identity:get_group(group_id=d00b2f341ffd4c80afe10b8079f151e3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:19.440446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2877ca8-4109-449e-9dcf-e2e21320b3f6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:19.442159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 224/454] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 4 05:41:19 2022] GET /identity/v3/groups/d00b2f341ffd4c80afe10b8079f151e3 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:19.448846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e9587-bb53-45f1-90cd-2a697e1146ae None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:19.541239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:19.564967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e9587-bb53-45f1-90cd-2a697e1146ae None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:19.601866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b4de7-c4bf-4177-9c27-a48ad737ab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:19.602527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b4de7-c4bf-4177-9c27-a48ad737ab96 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:19.602807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b4de7-c4bf-4177-9c27-a48ad737ab96 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:19.603083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b4de7-c4bf-4177-9c27-a48ad737ab96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:19.604150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b4de7-c4bf-4177-9c27-a48ad737ab96 None admin] RBAC: Adding query filter params (group.id=d00b2f341ffd4c80afe10b8079f151e3, role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=8a4d1bfa73f141cd90a91a1d9547a701) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:19.604464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b4de7-c4bf-4177-9c27-a48ad737ab96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:19.605906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b4de7-c4bf-4177-9c27-a48ad737ab96 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:19.612283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 231/455] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 4 05:41:19 2022] GET /identity/v3/role_assignments?group.id=d00b2f341ffd4c80afe10b8079f151e3&scope.project.id=8a4d1bfa73f141cd90a91a1d9547a701&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:21.981677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03291524-e0d2-45b1-82db-a5d99e601748 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:21.982709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03291524-e0d2-45b1-82db-a5d99e601748 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:21.983210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03291524-e0d2-45b1-82db-a5d99e601748 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:21.985115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 225/456] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:21.990281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928dd5d-dbd0-4bd2-bf07-8675d844da63 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:21.991080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928dd5d-dbd0-4bd2-bf07-8675d844da63 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:21.991607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928dd5d-dbd0-4bd2-bf07-8675d844da63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.017827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d928dd5d-dbd0-4bd2-bf07-8675d844da63 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:22.019414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d928dd5d-dbd0-4bd2-bf07-8675d844da63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:22.105635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 232/457] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:22.154045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03291524-e0d2-45b1-82db-a5d99e601748 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:22.244973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:22.290235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03291524-e0d2-45b1-82db-a5d99e601748 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:22.335493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3c3ef-613d-47cd-900b-b743f10ec088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:22.336665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3c3ef-613d-47cd-900b-b743f10ec088 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:22.337192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3c3ef-613d-47cd-900b-b743f10ec088 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:22.337688 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3c3ef-613d-47cd-900b-b743f10ec088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.348439 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3c3ef-613d-47cd-900b-b743f10ec088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:22.350183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3c3ef-613d-47cd-900b-b743f10ec088 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-12c3c3ef-613d-47cd-900b-b743f10ec088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:22.373592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:22.374501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:22.375381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:22.375381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:22.375381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:22.375381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:22.377371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 226/458] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:22.383772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928dd5d-dbd0-4bd2-bf07-8675d844da63 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:22.455390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:22.494218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928dd5d-dbd0-4bd2-bf07-8675d844da63 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:22.542016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4494a1-cfcc-4e44-9db3-50f14542738c None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:22.542329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4494a1-cfcc-4e44-9db3-50f14542738c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:22.542573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4494a1-cfcc-4e44-9db3-50f14542738c None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.543989 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 227/459] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:22.545870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b32bcd-ba66-4317-825f-caaf4738ce26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:22.546546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32bcd-ba66-4317-825f-caaf4738ce26 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:22.546827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32bcd-ba66-4317-825f-caaf4738ce26 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:22.547103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32bcd-ba66-4317-825f-caaf4738ce26 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.547966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b32bcd-ba66-4317-825f-caaf4738ce26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:22.548288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b32bcd-ba66-4317-825f-caaf4738ce26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:22.549486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812af6af-9192-4dc9-b607-b11bd0d9b701 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:22.549661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b32bcd-ba66-4317-825f-caaf4738ce26 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:22.550132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812af6af-9192-4dc9-b607-b11bd0d9b701 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:22.550589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812af6af-9192-4dc9-b607-b11bd0d9b701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.572734 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-812af6af-9192-4dc9-b607-b11bd0d9b701 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:22.573937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-812af6af-9192-4dc9-b607-b11bd0d9b701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:22.575619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 233/460] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:22.580984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b32bcd-ba66-4317-825f-caaf4738ce26 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:22.648790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 228/461] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:22.692561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812af6af-9192-4dc9-b607-b11bd0d9b701 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:22.724667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b32bcd-ba66-4317-825f-caaf4738ce26 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:22.754814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:22.759284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:22.765843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10344f90-3336-42da-9e6c-1c8d6cef9ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:22.766853 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10344f90-3336-42da-9e6c-1c8d6cef9ff6 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:22.767574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10344f90-3336-42da-9e6c-1c8d6cef9ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:22.768135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10344f90-3336-42da-9e6c-1c8d6cef9ff6 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.769054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10344f90-3336-42da-9e6c-1c8d6cef9ff6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:22.770824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10344f90-3336-42da-9e6c-1c8d6cef9ff6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:22.797555 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 234/462] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:41:22 2022] POST /identity/v3/users => generated 246 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:41:22.799518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7482abf-8c8d-4206-803a-5e393def0fd8 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:22.800001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7482abf-8c8d-4206-803a-5e393def0fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:22.800543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7482abf-8c8d-4206-803a-5e393def0fd8 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.806892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 235/463] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:22.808832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931280-53b9-467d-aa93-903a21174652 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:22.809408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931280-53b9-467d-aa93-903a21174652 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:22.809960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931280-53b9-467d-aa93-903a21174652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.813145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812af6af-9192-4dc9-b607-b11bd0d9b701 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:22.829762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d4931280-53b9-467d-aa93-903a21174652 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:22.831540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d4931280-53b9-467d-aa93-903a21174652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:22.853272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eac785-8041-4731-b386-ff6a8c149195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:22.854122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac785-8041-4731-b386-ff6a8c149195 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:22.854401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac785-8041-4731-b386-ff6a8c149195 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:22.854673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac785-8041-4731-b386-ff6a8c149195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.866966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eac785-8041-4731-b386-ff6a8c149195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:22.868602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eac785-8041-4731-b386-ff6a8c149195 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-64eac785-8041-4731-b386-ff6a8c149195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:22.889552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:22.890520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:22.891384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:22.891384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:22.891384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:22.891384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:22.891384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 229/464] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:22.893245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988def0-a752-4310-891d-7ba06df8eaa8 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:22.893378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988def0-a752-4310-891d-7ba06df8eaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:22.893637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988def0-a752-4310-891d-7ba06df8eaa8 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.895389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 230/465] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:22.900943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c531d1-d348-4328-a27e-39b44b5ff420 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:22.901396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c531d1-d348-4328-a27e-39b44b5ff420 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:22.902291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c531d1-d348-4328-a27e-39b44b5ff420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:22.917459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 236/466] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:22.923287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-e2c531d1-d348-4328-a27e-39b44b5ff420 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:22.923287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e2c531d1-d348-4328-a27e-39b44b5ff420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:22.924623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4931280-53b9-467d-aa93-903a21174652 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:23.003884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 231/467] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:23.004800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c531d1-d348-4328-a27e-39b44b5ff420 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:23.039520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:23.071896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4931280-53b9-467d-aa93-903a21174652 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:23.094208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:23.120078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c531d1-d348-4328-a27e-39b44b5ff420 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:23.124229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cc2ff-adb0-4599-b996-1d13dec93db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:23.124914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cc2ff-adb0-4599-b996-1d13dec93db2 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.125194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cc2ff-adb0-4599-b996-1d13dec93db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.125472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cc2ff-adb0-4599-b996-1d13dec93db2 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.126211 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cc2ff-adb0-4599-b996-1d13dec93db2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:23.126515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cc2ff-adb0-4599-b996-1d13dec93db2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:23.128485 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cc2ff-adb0-4599-b996-1d13dec93db2 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:23.145201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 237/468] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:23.145863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cc2ff-adb0-4599-b996-1d13dec93db2 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:23.165476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ec033-d042-4d56-8db0-709794228ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:23.166785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ec033-d042-4d56-8db0-709794228ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.167071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ec033-d042-4d56-8db0-709794228ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.167378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ec033-d042-4d56-8db0-709794228ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.183098 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ec033-d042-4d56-8db0-709794228ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:23.184704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ec033-d042-4d56-8db0-709794228ed7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-1b7ec033-d042-4d56-8db0-709794228ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:23.205858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:23.206955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:23.207843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:23.207843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:23.207843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:23.207843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:23.207843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 232/469] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:23.208545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ec033-d042-4d56-8db0-709794228ed7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:23.281400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cc2ff-adb0-4599-b996-1d13dec93db2 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:23.318990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7c3ae-22e6-4125-a50a-d552d0685334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:23.320106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7c3ae-22e6-4125-a50a-d552d0685334 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.320387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7c3ae-22e6-4125-a50a-d552d0685334 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.320657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7c3ae-22e6-4125-a50a-d552d0685334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.324823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:23.332290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7c3ae-22e6-4125-a50a-d552d0685334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:23.333708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7c3ae-22e6-4125-a50a-d552d0685334 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:23.346580 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ec033-d042-4d56-8db0-709794228ed7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-eff7c3ae-22e6-4125-a50a-d552d0685334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:23.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:23.353020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:23.353927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:23.353927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:23.353927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:23.353927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:23.353927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 238/470] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:23.354653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7c3ae-22e6-4125-a50a-d552d0685334 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:23.387077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:23.399542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5799c7-7569-4b12-aa9b-42750fb99ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:23.400539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5799c7-7569-4b12-aa9b-42750fb99ca0 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.400824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5799c7-7569-4b12-aa9b-42750fb99ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.401101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5799c7-7569-4b12-aa9b-42750fb99ca0 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.402186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5799c7-7569-4b12-aa9b-42750fb99ca0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:23.404028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5799c7-7569-4b12-aa9b-42750fb99ca0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:23.431665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 233/471] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:41:23 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:41:23.433835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5799c7-7569-4b12-aa9b-42750fb99ca0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:23.467713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7c3ae-22e6-4125-a50a-d552d0685334 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:23.511836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc764d-0322-4259-8d2b-dcc9a8332949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:23.512688 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc764d-0322-4259-8d2b-dcc9a8332949 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.513086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc764d-0322-4259-8d2b-dcc9a8332949 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.513358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc764d-0322-4259-8d2b-dcc9a8332949 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.514094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc764d-0322-4259-8d2b-dcc9a8332949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:23.514394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc764d-0322-4259-8d2b-dcc9a8332949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:23.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc764d-0322-4259-8d2b-dcc9a8332949 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:23.534716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 239/472] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:23.536476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9e149-99b4-44b6-80ad-42e539fbd763 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.537064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9e149-99b4-44b6-80ad-42e539fbd763 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.537596 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9e149-99b4-44b6-80ad-42e539fbd763 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.543664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 240/473] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:23.544908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9e149-99b4-44b6-80ad-42e539fbd763 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:23.561439 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5799c7-7569-4b12-aa9b-42750fb99ca0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:23.601118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1203b-32c3-42a9-8fa8-813a84c18304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:23.601732 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1203b-32c3-42a9-8fa8-813a84c18304 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.602525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1203b-32c3-42a9-8fa8-813a84c18304 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.602829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1203b-32c3-42a9-8fa8-813a84c18304 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.603677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1203b-32c3-42a9-8fa8-813a84c18304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:23.604003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1203b-32c3-42a9-8fa8-813a84c18304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:23.605970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1203b-32c3-42a9-8fa8-813a84c18304 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:23.626702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:23.632245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 234/474] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:23.633692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1f2c-7606-4ada-b9bc-2224c847372e None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.633987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1f2c-7606-4ada-b9bc-2224c847372e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.634270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1f2c-7606-4ada-b9bc-2224c847372e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.655543 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c98d1f2c-7606-4ada-b9bc-2224c847372e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:23.656991 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c98d1f2c-7606-4ada-b9bc-2224c847372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:23.657809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:23.669762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9e149-99b4-44b6-80ad-42e539fbd763 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:23.718574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463504f8-7bdc-4dd8-8492-0be7db7a3bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:23.719285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463504f8-7bdc-4dd8-8492-0be7db7a3bf0 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.719565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463504f8-7bdc-4dd8-8492-0be7db7a3bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.719834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463504f8-7bdc-4dd8-8492-0be7db7a3bf0 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.721055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463504f8-7bdc-4dd8-8492-0be7db7a3bf0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:23.721990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463504f8-7bdc-4dd8-8492-0be7db7a3bf0 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:23.740343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 235/475] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:23.741929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d1f2c-7606-4ada-b9bc-2224c847372e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:23.744804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 241/476] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:41:23 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:41:23.768231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463504f8-7bdc-4dd8-8492-0be7db7a3bf0 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:23.902605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463504f8-7bdc-4dd8-8492-0be7db7a3bf0 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:23.922060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d1f2c-7606-4ada-b9bc-2224c847372e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:23.968127 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe1bb0-8c76-4409-be80-6703328eb52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:23.969091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe1bb0-8c76-4409-be80-6703328eb52e None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.969374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe1bb0-8c76-4409-be80-6703328eb52e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.969648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe1bb0-8c76-4409-be80-6703328eb52e None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.970493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:23.971422 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe1bb0-8c76-4409-be80-6703328eb52e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:23.973912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe1bb0-8c76-4409-be80-6703328eb52e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:23.977906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ad417-12d4-429e-824a-e311c2216c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:23.979052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ad417-12d4-429e-824a-e311c2216c93 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:23.979586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ad417-12d4-429e-824a-e311c2216c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:23.979958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ad417-12d4-429e-824a-e311c2216c93 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:23.980789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ad417-12d4-429e-824a-e311c2216c93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:23.981826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.sql.core [None req-c2fe1bb0-8c76-4409-be80-6703328eb52e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 05:41:23.981826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 05:41:23.981826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [parameters: {'id': 'dfa3953c7d0142ab8b379ac70c56cf8e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 05:41:23.981826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80128) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 05:41:23.982764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ad417-12d4-429e-824a-e311c2216c93 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-c2fe1bb0-8c76-4409-be80-6703328eb52e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application conn.query(q) Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application result.read() Jul 04 05:41:24.001532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return ref.to_dict() Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 05:41:24.002529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application next(self.gen) Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application yield resource Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application next(self.gen) Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise self.value Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application session.commit() Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 05:41:24.003441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self.session.flush() Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application compat.raise_( Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 05:41:24.004276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 05:41:24.005124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application conn.query(q) Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application result.read() Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 05:41:24.005938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [parameters: {'id': 'dfa3953c7d0142ab8b379ac70c56cf8e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:24.006700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 05:41:24.009459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:24.011280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 242/477] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jul 4 05:41:23 2022] POST /identity/v3/roles => generated 144 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:24.011280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe1bb0-8c76-4409-be80-6703328eb52e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:24.025364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:24.029282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 236/478] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:41:23 2022] POST /identity/v3/users => generated 249 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:41:24.150436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe1bb0-8c76-4409-be80-6703328eb52e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:24.189668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48c13b-83d7-4160-aed6-f5323d12cb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:24.190833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48c13b-83d7-4160-aed6-f5323d12cb3c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:24.191346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48c13b-83d7-4160-aed6-f5323d12cb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:24.191831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48c13b-83d7-4160-aed6-f5323d12cb3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:24.205469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48c13b-83d7-4160-aed6-f5323d12cb3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-ff48c13b-83d7-4160-aed6-f5323d12cb3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:24.214563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:24.215505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 05:41:24.216300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:24.219125 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 243/479] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:41:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:24.225052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ad417-12d4-429e-824a-e311c2216c93 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:24.330416 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ad417-12d4-429e-824a-e311c2216c93 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:24.364493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f3fd8-7a03-4142-8e70-f4da7b3cce2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:24.365524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f3fd8-7a03-4142-8e70-f4da7b3cce2d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:24.366000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f3fd8-7a03-4142-8e70-f4da7b3cce2d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:24.366464 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f3fd8-7a03-4142-8e70-f4da7b3cce2d None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:24.367458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f3fd8-7a03-4142-8e70-f4da7b3cce2d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:24.367974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f3fd8-7a03-4142-8e70-f4da7b3cce2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:24.369680 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f3fd8-7a03-4142-8e70-f4da7b3cce2d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:24.378818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 237/480] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:25.213004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc783daf-6a9d-4bfb-afa2-f234df7265a6 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:25.213530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc783daf-6a9d-4bfb-afa2-f234df7265a6 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:25.213827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc783daf-6a9d-4bfb-afa2-f234df7265a6 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:25.215368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 244/481] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:25.220716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1ad21-0815-4f78-9baf-c58b91e68031 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:25.221131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1ad21-0815-4f78-9baf-c58b91e68031 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:25.221298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1ad21-0815-4f78-9baf-c58b91e68031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:25.241584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d0b1ad21-0815-4f78-9baf-c58b91e68031 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:25.242789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d0b1ad21-0815-4f78-9baf-c58b91e68031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:25.286933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:25.329308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 238/482] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:25.365572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc783daf-6a9d-4bfb-afa2-f234df7265a6 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:25.456155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:25.496087 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc783daf-6a9d-4bfb-afa2-f234df7265a6 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:25.542144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83f01a-4f3c-45bf-91eb-5ad4ad1050ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:25.542991 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83f01a-4f3c-45bf-91eb-5ad4ad1050ed None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:25.543290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83f01a-4f3c-45bf-91eb-5ad4ad1050ed None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:25.543567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83f01a-4f3c-45bf-91eb-5ad4ad1050ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:25.558886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83f01a-4f3c-45bf-91eb-5ad4ad1050ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-5e83f01a-4f3c-45bf-91eb-5ad4ad1050ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:25.566788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:25.567768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:25.568562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:25.574668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1ad21-0815-4f78-9baf-c58b91e68031 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:25.576881 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 245/483] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:25.712094 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1ad21-0815-4f78-9baf-c58b91e68031 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:25.766762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0b94e-ad80-401a-a249-60d205983c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:25.767847 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0b94e-ad80-401a-a249-60d205983c55 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:25.768335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0b94e-ad80-401a-a249-60d205983c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:25.768793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0b94e-ad80-401a-a249-60d205983c55 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:25.769757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0b94e-ad80-401a-a249-60d205983c55 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:25.770248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0b94e-ad80-401a-a249-60d205983c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:25.771843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0b94e-ad80-401a-a249-60d205983c55 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:25.781015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 239/484] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:25.786544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83f01a-4f3c-45bf-91eb-5ad4ad1050ed None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:25.924796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83f01a-4f3c-45bf-91eb-5ad4ad1050ed None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:25.963793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:25.971874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0dd7c-a1a2-4ec6-a51e-d2dcbf97fc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:25.972686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0dd7c-a1a2-4ec6-a51e-d2dcbf97fc48 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:25.972969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0dd7c-a1a2-4ec6-a51e-d2dcbf97fc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:25.973239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0dd7c-a1a2-4ec6-a51e-d2dcbf97fc48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:25.986989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0dd7c-a1a2-4ec6-a51e-d2dcbf97fc48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:25.988495 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0dd7c-a1a2-4ec6-a51e-d2dcbf97fc48 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-70c0dd7c-a1a2-4ec6-a51e-d2dcbf97fc48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:26.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:26.006530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:26.007416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:26.007416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:26.007416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:26.007416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:26.007416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 246/485] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:26.012598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0b94e-ad80-401a-a249-60d205983c55 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:26.029015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bb28c-fbbc-4ff2-9822-95778a40a23f None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.029274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bb28c-fbbc-4ff2-9822-95778a40a23f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.029501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bb28c-fbbc-4ff2-9822-95778a40a23f None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.031138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 247/486] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:41:26.039272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524705f8-1931-4d00-bf42-7e634abe5a23 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.039272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524705f8-1931-4d00-bf42-7e634abe5a23 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.039272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524705f8-1931-4d00-bf42-7e634abe5a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.064135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-524705f8-1931-4d00-bf42-7e634abe5a23 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:26.068327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-524705f8-1931-4d00-bf42-7e634abe5a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:26.100813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:26.168778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0b94e-ad80-401a-a249-60d205983c55 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:26.222199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 248/487] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:26.234497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc445d-439f-4448-b872-55be57d48721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:26.235185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc445d-439f-4448-b872-55be57d48721 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.235493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc445d-439f-4448-b872-55be57d48721 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.235766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc445d-439f-4448-b872-55be57d48721 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.236501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc445d-439f-4448-b872-55be57d48721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:26.236806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc445d-439f-4448-b872-55be57d48721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:26.238199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc445d-439f-4448-b872-55be57d48721 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:26.251698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 240/488] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:26.256919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524705f8-1931-4d00-bf42-7e634abe5a23 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:26.266745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc445d-439f-4448-b872-55be57d48721 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:26.423351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:26.423351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc445d-439f-4448-b872-55be57d48721 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:26.451753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524705f8-1931-4d00-bf42-7e634abe5a23 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:26.483711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:26.488571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f3dd4-fe3e-4cd4-b45a-5b74675f810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:26.489398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f3dd4-fe3e-4cd4-b45a-5b74675f810a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.489677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f3dd4-fe3e-4cd4-b45a-5b74675f810a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.489945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f3dd4-fe3e-4cd4-b45a-5b74675f810a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.520511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f3dd4-fe3e-4cd4-b45a-5b74675f810a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:26.526399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd125a7b-967f-489a-8e60-44180a68d3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:26.527212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd125a7b-967f-489a-8e60-44180a68d3ba None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.527517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd125a7b-967f-489a-8e60-44180a68d3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.527788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd125a7b-967f-489a-8e60-44180a68d3ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-835f3dd4-fe3e-4cd4-b45a-5b74675f810a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:26.533264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:26.534152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:26.534946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:26.540953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 241/489] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:26.542223 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f3dd4-fe3e-4cd4-b45a-5b74675f810a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:26.543961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd125a7b-967f-489a-8e60-44180a68d3ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:26.545438 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd125a7b-967f-489a-8e60-44180a68d3ba None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-cd125a7b-967f-489a-8e60-44180a68d3ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:26.571347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:26.573509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:26.575417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 249/490] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 4 05:41:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:26.578459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd125a7b-967f-489a-8e60-44180a68d3ba None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:26.680761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f3dd4-fe3e-4cd4-b45a-5b74675f810a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:26.714579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd125a7b-967f-489a-8e60-44180a68d3ba None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:26.715398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca4d0-b442-4b6a-95a1-5c5a788f4367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:26.716211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca4d0-b442-4b6a-95a1-5c5a788f4367 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.716565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca4d0-b442-4b6a-95a1-5c5a788f4367 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.716902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca4d0-b442-4b6a-95a1-5c5a788f4367 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.717727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ca4d0-b442-4b6a-95a1-5c5a788f4367 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:26.718103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ca4d0-b442-4b6a-95a1-5c5a788f4367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:26.719602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ca4d0-b442-4b6a-95a1-5c5a788f4367 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:26.728690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 242/491] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:26.730272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323970e-470d-448d-9d46-b9c6e145f288 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.730630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323970e-470d-448d-9d46-b9c6e145f288 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.730975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323970e-470d-448d-9d46-b9c6e145f288 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.732557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 243/492] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:26.734645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323970e-470d-448d-9d46-b9c6e145f288 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:26.755235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b4833-2193-4239-81e2-826d312f645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:26.755946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4833-2193-4239-81e2-826d312f645c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.756230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4833-2193-4239-81e2-826d312f645c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.756506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4833-2193-4239-81e2-826d312f645c None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.757352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b4833-2193-4239-81e2-826d312f645c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:26.757660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b4833-2193-4239-81e2-826d312f645c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:26.759141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b4833-2193-4239-81e2-826d312f645c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:26.774356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:26.798477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 250/493] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 4 05:41:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:26.799834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f5fe-95e5-432d-ae94-6d9b76536446 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.800121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f5fe-95e5-432d-ae94-6d9b76536446 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.800396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f5fe-95e5-432d-ae94-6d9b76536446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.816850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-7292f5fe-95e5-432d-ae94-6d9b76536446 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:26.818053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7292f5fe-95e5-432d-ae94-6d9b76536446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:26.844708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323970e-470d-448d-9d46-b9c6e145f288 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:26.845174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:26.881549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eafce-5450-4947-9b3e-6a8dd8054b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:26.882355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eafce-5450-4947-9b3e-6a8dd8054b94 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.882638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eafce-5450-4947-9b3e-6a8dd8054b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.882899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eafce-5450-4947-9b3e-6a8dd8054b94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.890882 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 251/494] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:26.892242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c488d-4e67-4264-945d-5827c07c555d None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.892528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c488d-4e67-4264-945d-5827c07c555d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.892802 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c488d-4e67-4264-945d-5827c07c555d None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.893711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eafce-5450-4947-9b3e-6a8dd8054b94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:26.895352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 252/495] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:26.895352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c488d-4e67-4264-945d-5827c07c555d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:26.895569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eafce-5450-4947-9b3e-6a8dd8054b94 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-790eafce-5450-4947-9b3e-6a8dd8054b94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:26.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:26.915833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:26.916755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:26.916755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:26.916755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:26.916755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:26.916755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 244/496] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:26.917577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee3b3-f57d-4c89-874c-fe5aa109fd86 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.917862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee3b3-f57d-4c89-874c-fe5aa109fd86 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.918132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee3b3-f57d-4c89-874c-fe5aa109fd86 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.923716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 245/497] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:26.923716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a685-ba83-405d-bb31-3e7f04f67e2f None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:26.923716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a685-ba83-405d-bb31-3e7f04f67e2f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:26.923716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a685-ba83-405d-bb31-3e7f04f67e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:26.938607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2256a685-ba83-405d-bb31-3e7f04f67e2f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:26.939962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2256a685-ba83-405d-bb31-3e7f04f67e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:27.004380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c488d-4e67-4264-945d-5827c07c555d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.004700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 246/498] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:27.005720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256a685-ba83-405d-bb31-3e7f04f67e2f None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.017481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:27.039736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01bbb8-0ab8-448a-b4b0-8257ef2c168f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.040386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01bbb8-0ab8-448a-b4b0-8257ef2c168f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.040678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01bbb8-0ab8-448a-b4b0-8257ef2c168f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.040982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01bbb8-0ab8-448a-b4b0-8257ef2c168f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.067472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:27.108441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01bbb8-0ab8-448a-b4b0-8257ef2c168f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:27.109943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01bbb8-0ab8-448a-b4b0-8257ef2c168f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:27.111259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256a685-ba83-405d-bb31-3e7f04f67e2f None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.143097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883b4db-f234-4473-9ead-4cd85762e484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.143926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b4db-f234-4473-9ead-4cd85762e484 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.144202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b4db-f234-4473-9ead-4cd85762e484 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.144469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b4db-f234-4473-9ead-4cd85762e484 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.154389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883b4db-f234-4473-9ead-4cd85762e484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-3883b4db-f234-4473-9ead-4cd85762e484 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:27.162051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:27.162953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:27.163877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 247/499] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:27.164542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883b4db-f234-4473-9ead-4cd85762e484 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.213755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 253/500] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:27.215097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73da74e-6067-4f5f-8075-e3501583b79d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.215787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73da74e-6067-4f5f-8075-e3501583b79d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.215787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73da74e-6067-4f5f-8075-e3501583b79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.231624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d73da74e-6067-4f5f-8075-e3501583b79d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:27.232795 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d73da74e-6067-4f5f-8075-e3501583b79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:27.233911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:27.272622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883b4db-f234-4473-9ead-4cd85762e484 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.287383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:27.299027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 254/501] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:27.299871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73da74e-6067-4f5f-8075-e3501583b79d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.306998 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35e6bb-aacd-48b9-b92a-6db578f1a7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.308121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35e6bb-aacd-48b9-b92a-6db578f1a7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.308401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35e6bb-aacd-48b9-b92a-6db578f1a7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.308671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35e6bb-aacd-48b9-b92a-6db578f1a7e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.309729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35e6bb-aacd-48b9-b92a-6db578f1a7e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:27.310056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35e6bb-aacd-48b9-b92a-6db578f1a7e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:27.312040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35e6bb-aacd-48b9-b92a-6db578f1a7e6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:27.325165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 248/502] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:27.325971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35e6bb-aacd-48b9-b92a-6db578f1a7e6 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.399916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73da74e-6067-4f5f-8075-e3501583b79d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.421220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35e6bb-aacd-48b9-b92a-6db578f1a7e6 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.435529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74069b0f-178a-43d1-9352-7607d44bbd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.436320 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74069b0f-178a-43d1-9352-7607d44bbd3d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.436593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74069b0f-178a-43d1-9352-7607d44bbd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.436863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74069b0f-178a-43d1-9352-7607d44bbd3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.444646 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:27.447679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74069b0f-178a-43d1-9352-7607d44bbd3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:27.449101 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74069b0f-178a-43d1-9352-7607d44bbd3d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:27.455762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7183b15-e593-4813-87fd-30aaf1600984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.456444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7183b15-e593-4813-87fd-30aaf1600984 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.456726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7183b15-e593-4813-87fd-30aaf1600984 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.456998 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7183b15-e593-4813-87fd-30aaf1600984 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.457748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7183b15-e593-4813-87fd-30aaf1600984 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:27.458050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7183b15-e593-4813-87fd-30aaf1600984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:27.459435 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7183b15-e593-4813-87fd-30aaf1600984 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-74069b0f-178a-43d1-9352-7607d44bbd3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:27.466352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:27.467513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:27.468368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:27.468368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:27.468368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:27.468368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:27.468368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 255/503] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:27.468368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74069b0f-178a-43d1-9352-7607d44bbd3d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.471396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 249/504] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:27.472508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7183b15-e593-4813-87fd-30aaf1600984 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.519029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:27.573131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7183b15-e593-4813-87fd-30aaf1600984 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.576513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74069b0f-178a-43d1-9352-7607d44bbd3d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.608342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947175d1-080a-41b2-9e89-77ceb9043974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.609143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947175d1-080a-41b2-9e89-77ceb9043974 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.609428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947175d1-080a-41b2-9e89-77ceb9043974 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.609690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947175d1-080a-41b2-9e89-77ceb9043974 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.611811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379d502-3244-40a8-b3ec-1d5edcf7981b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.612646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379d502-3244-40a8-b3ec-1d5edcf7981b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.612927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379d502-3244-40a8-b3ec-1d5edcf7981b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.613202 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379d502-3244-40a8-b3ec-1d5edcf7981b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.619504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947175d1-080a-41b2-9e89-77ceb9043974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:27.623497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379d502-3244-40a8-b3ec-1d5edcf7981b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:27.624985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379d502-3244-40a8-b3ec-1d5edcf7981b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-947175d1-080a-41b2-9e89-77ceb9043974 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:27.627112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:27.628077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:27.628883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 250/505] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:27.629534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947175d1-080a-41b2-9e89-77ceb9043974 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-3379d502-3244-40a8-b3ec-1d5edcf7981b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:27.642552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:27.643474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:27.644474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:27.644474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:27.644474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:27.644474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:27.644474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 256/506] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:27.645084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379d502-3244-40a8-b3ec-1d5edcf7981b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.679433 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:27.734415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947175d1-080a-41b2-9e89-77ceb9043974 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.741423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:27.749008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379d502-3244-40a8-b3ec-1d5edcf7981b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.768117 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b914e9-47a4-4b6c-b5c0-f0c0dc790ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.768926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b914e9-47a4-4b6c-b5c0-f0c0dc790ebe None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.769202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b914e9-47a4-4b6c-b5c0-f0c0dc790ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.769512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b914e9-47a4-4b6c-b5c0-f0c0dc790ebe None admin] PATH_INFO: `/v3/users/glance` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.777017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b914e9-47a4-4b6c-b5c0-f0c0dc790ebe None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:27.778964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b914e9-47a4-4b6c-b5c0-f0c0dc790ebe None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:27.783192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba4595-df1b-46fc-ab23-280e31f75eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.784012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba4595-df1b-46fc-ab23-280e31f75eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.784290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba4595-df1b-46fc-ab23-280e31f75eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.784558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba4595-df1b-46fc-ab23-280e31f75eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-a7b914e9-47a4-4b6c-b5c0-f0c0dc790ebe None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:27.794372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:27.795472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:27.801248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:27.801248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:27.801248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:27.801248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:27.801248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:27.801248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:27.801248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:27.801248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:27.801248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 05:41:27.801248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:27.801764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba4595-df1b-46fc-ab23-280e31f75eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:27.801834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 251/507] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:27.801899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba4595-df1b-46fc-ab23-280e31f75eb8 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:27.801964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b914e9-47a4-4b6c-b5c0-f0c0dc790ebe None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-19ba4595-df1b-46fc-ab23-280e31f75eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:27.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:27.819181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:27.820413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:27.820413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:27.820413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:27.820413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:27.820413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 257/508] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:27.820990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba4595-df1b-46fc-ab23-280e31f75eb8 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.904358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b914e9-47a4-4b6c-b5c0-f0c0dc790ebe None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.911105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:27.919401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba4595-df1b-46fc-ab23-280e31f75eb8 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:27.938467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9ef7e-d465-4096-83cb-0cddd6d49290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.939125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9ef7e-d465-4096-83cb-0cddd6d49290 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.939454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9ef7e-d465-4096-83cb-0cddd6d49290 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.939720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9ef7e-d465-4096-83cb-0cddd6d49290 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.940440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9ef7e-d465-4096-83cb-0cddd6d49290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:27.940726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9ef7e-d465-4096-83cb-0cddd6d49290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:27.942108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9ef7e-d465-4096-83cb-0cddd6d49290 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:27.953275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1e2ac-847c-4a9d-83df-ec7633a39bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:27.953434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 252/509] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:27.953945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1e2ac-847c-4a9d-83df-ec7633a39bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:27.954055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9ef7e-d465-4096-83cb-0cddd6d49290 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.954225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1e2ac-847c-4a9d-83df-ec7633a39bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:27.954570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1e2ac-847c-4a9d-83df-ec7633a39bb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:27.955344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1e2ac-847c-4a9d-83df-ec7633a39bb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:27.955652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1e2ac-847c-4a9d-83df-ec7633a39bb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:27.957043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1e2ac-847c-4a9d-83df-ec7633a39bb3 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:27.965229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 258/510] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:27.966020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1e2ac-847c-4a9d-83df-ec7633a39bb3 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:27.976718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:28.062366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9ef7e-d465-4096-83cb-0cddd6d49290 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.069118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1e2ac-847c-4a9d-83df-ec7633a39bb3 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.103995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d82d5-743a-429f-bcfd-43a0819a6af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.104663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d82d5-743a-429f-bcfd-43a0819a6af1 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.104906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2fbec-409e-4cdb-af60-3b468a84732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.105018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d82d5-743a-429f-bcfd-43a0819a6af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.105196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d82d5-743a-429f-bcfd-43a0819a6af1 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.105578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2fbec-409e-4cdb-af60-3b468a84732b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.105858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2fbec-409e-4cdb-af60-3b468a84732b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.106026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d82d5-743a-429f-bcfd-43a0819a6af1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:28.106156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2fbec-409e-4cdb-af60-3b468a84732b None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.106339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d82d5-743a-429f-bcfd-43a0819a6af1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.106860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2fbec-409e-4cdb-af60-3b468a84732b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:28.107163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2fbec-409e-4cdb-af60-3b468a84732b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.107824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d82d5-743a-429f-bcfd-43a0819a6af1 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.108563 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2fbec-409e-4cdb-af60-3b468a84732b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.120333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 253/511] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:28.121140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2fbec-409e-4cdb-af60-3b468a84732b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:28.125652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 259/512] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 4 05:41:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:28.126449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d82d5-743a-429f-bcfd-43a0819a6af1 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:28.154050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:28.191177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:28.226865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2fbec-409e-4cdb-af60-3b468a84732b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.227957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d82d5-743a-429f-bcfd-43a0819a6af1 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.260456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493bb2c-fe6e-454b-aeb1-1a9cc837ef78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.261144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493bb2c-fe6e-454b-aeb1-1a9cc837ef78 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.261418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493bb2c-fe6e-454b-aeb1-1a9cc837ef78 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.261690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493bb2c-fe6e-454b-aeb1-1a9cc837ef78 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.262417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493bb2c-fe6e-454b-aeb1-1a9cc837ef78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:28.262713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493bb2c-fe6e-454b-aeb1-1a9cc837ef78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.264119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493bb2c-fe6e-454b-aeb1-1a9cc837ef78 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.264359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78578754-6110-403c-9a01-4c0c1d8f4233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.265031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578754-6110-403c-9a01-4c0c1d8f4233 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.265296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578754-6110-403c-9a01-4c0c1d8f4233 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.265584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578754-6110-403c-9a01-4c0c1d8f4233 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.266283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78578754-6110-403c-9a01-4c0c1d8f4233 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.267745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78578754-6110-403c-9a01-4c0c1d8f4233 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.275141 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 254/513] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:28.275964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493bb2c-fe6e-454b-aeb1-1a9cc837ef78 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:28.290140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 260/514] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:41:28 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:41:28.291051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78578754-6110-403c-9a01-4c0c1d8f4233 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:28.374290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:28.391641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493bb2c-fe6e-454b-aeb1-1a9cc837ef78 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.416077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78578754-6110-403c-9a01-4c0c1d8f4233 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.430359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e63be-c2d4-4a19-adc8-562334df2ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.431176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e63be-c2d4-4a19-adc8-562334df2ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.431521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e63be-c2d4-4a19-adc8-562334df2ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.431800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e63be-c2d4-4a19-adc8-562334df2ee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.441678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:28.444793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e63be-c2d4-4a19-adc8-562334df2ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.446289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e63be-c2d4-4a19-adc8-562334df2ee2 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.457116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709c2f1-dafa-4060-9fbc-d6bf8e518e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.458059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c2f1-dafa-4060-9fbc-d6bf8e518e16 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.458322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c2f1-dafa-4060-9fbc-d6bf8e518e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.458593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c2f1-dafa-4060-9fbc-d6bf8e518e16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-cc3e63be-c2d4-4a19-adc8-562334df2ee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:28.464806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:28.465633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:28.466670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:28.466670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:28.466670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:28.466670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:28.466873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709c2f1-dafa-4060-9fbc-d6bf8e518e16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.466941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 255/515] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:28.466941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e63be-c2d4-4a19-adc8-562334df2ee2 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:28.467075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709c2f1-dafa-4060-9fbc-d6bf8e518e16 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-b709c2f1-dafa-4060-9fbc-d6bf8e518e16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:28.485339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:28.486429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:28.487565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:28.487565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:28.487565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:28.487565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:28.487565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:28.487565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:28.487565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:28.487565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:28.487565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 261/516] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:28.488051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709c2f1-dafa-4060-9fbc-d6bf8e518e16 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:28.583288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e63be-c2d4-4a19-adc8-562334df2ee2 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.601032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709c2f1-dafa-4060-9fbc-d6bf8e518e16 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.619566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab22af-057a-4720-990b-7e5d585e0420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.620194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab22af-057a-4720-990b-7e5d585e0420 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.620492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab22af-057a-4720-990b-7e5d585e0420 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.620894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab22af-057a-4720-990b-7e5d585e0420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.630293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:28.636662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b167744-1dba-4dcf-be3b-6ef599f382c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.637471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b167744-1dba-4dcf-be3b-6ef599f382c1 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.637754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b167744-1dba-4dcf-be3b-6ef599f382c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.638037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b167744-1dba-4dcf-be3b-6ef599f382c1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.645650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b167744-1dba-4dcf-be3b-6ef599f382c1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.647115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b167744-1dba-4dcf-be3b-6ef599f382c1 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-5b167744-1dba-4dcf-be3b-6ef599f382c1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:28.663435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:28.664361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:28.665228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 262/517] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:28.665822 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b167744-1dba-4dcf-be3b-6ef599f382c1 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:28.698573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab22af-057a-4720-990b-7e5d585e0420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.700100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab22af-057a-4720-990b-7e5d585e0420 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.712838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:28.776185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b167744-1dba-4dcf-be3b-6ef599f382c1 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.805435 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 256/518] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:28.806313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab22af-057a-4720-990b-7e5d585e0420 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:28.817131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01d54-edaf-41fd-bb48-04cb37ac5f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.817800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01d54-edaf-41fd-bb48-04cb37ac5f68 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.818077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01d54-edaf-41fd-bb48-04cb37ac5f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.818346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01d54-edaf-41fd-bb48-04cb37ac5f68 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.819076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd01d54-edaf-41fd-bb48-04cb37ac5f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:28.819974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd01d54-edaf-41fd-bb48-04cb37ac5f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.821128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd01d54-edaf-41fd-bb48-04cb37ac5f68 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.833247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 263/519] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:28.834060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01d54-edaf-41fd-bb48-04cb37ac5f68 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:28.854789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:28.923988 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab22af-057a-4720-990b-7e5d585e0420 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.940190 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01d54-edaf-41fd-bb48-04cb37ac5f68 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:28.942057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:28.960011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c71307-ea2f-4b3a-bacd-acb143749831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.960676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c71307-ea2f-4b3a-bacd-acb143749831 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.960948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c71307-ea2f-4b3a-bacd-acb143749831 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.961214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c71307-ea2f-4b3a-bacd-acb143749831 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.961992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c71307-ea2f-4b3a-bacd-acb143749831 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:28.962762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c71307-ea2f-4b3a-bacd-acb143749831 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.964502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c71307-ea2f-4b3a-bacd-acb143749831 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.976061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570b3e9-b6ea-476e-bbce-966bbcb372a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:28.976743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570b3e9-b6ea-476e-bbce-966bbcb372a9 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:28.977023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570b3e9-b6ea-476e-bbce-966bbcb372a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:28.977294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570b3e9-b6ea-476e-bbce-966bbcb372a9 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:28.978127 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570b3e9-b6ea-476e-bbce-966bbcb372a9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:28.978532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570b3e9-b6ea-476e-bbce-966bbcb372a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:28.980123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570b3e9-b6ea-476e-bbce-966bbcb372a9 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:28.987305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 257/520] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:28.987305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c71307-ea2f-4b3a-bacd-acb143749831 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:29.003869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 264/521] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 4 05:41:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:29.009088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570b3e9-b6ea-476e-bbce-966bbcb372a9 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:29.083581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:29.087439 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c71307-ea2f-4b3a-bacd-acb143749831 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:29.111916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570b3e9-b6ea-476e-bbce-966bbcb372a9 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:29.124075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e70d01-cac2-4232-8f85-b88aae466b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:29.124883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e70d01-cac2-4232-8f85-b88aae466b1b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:29.125163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e70d01-cac2-4232-8f85-b88aae466b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:29.125434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e70d01-cac2-4232-8f85-b88aae466b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:29.135469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e70d01-cac2-4232-8f85-b88aae466b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:29.136896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e70d01-cac2-4232-8f85-b88aae466b1b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:29.148071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d97fd6-1e71-4bf1-810d-18e246af21c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:29.148873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d97fd6-1e71-4bf1-810d-18e246af21c6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:29.149155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d97fd6-1e71-4bf1-810d-18e246af21c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:29.149426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d97fd6-1e71-4bf1-810d-18e246af21c6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-e6e70d01-cac2-4232-8f85-b88aae466b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:29.154607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:29.155451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:29.156304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:29.156304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:29.156304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:29.156304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:29.156304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 258/522] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:29.157156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e70d01-cac2-4232-8f85-b88aae466b1b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:29.161201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d97fd6-1e71-4bf1-810d-18e246af21c6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:29.162695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d97fd6-1e71-4bf1-810d-18e246af21c6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-22d97fd6-1e71-4bf1-810d-18e246af21c6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:29.182653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:29.183755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:29.184658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 265/523] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:29.185777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d97fd6-1e71-4bf1-810d-18e246af21c6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:29.209969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:29.266909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e70d01-cac2-4232-8f85-b88aae466b1b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:29.290585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d97fd6-1e71-4bf1-810d-18e246af21c6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:29.303533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b8243-d899-406c-aa13-b68157d6b9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:29.304223 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b8243-d899-406c-aa13-b68157d6b9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:29.304503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b8243-d899-406c-aa13-b68157d6b9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:29.304849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b8243-d899-406c-aa13-b68157d6b9a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:29.305614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b8243-d899-406c-aa13-b68157d6b9a0 None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=2cb245e4300e4d5fbdd425e936f4ac77) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:29.306199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b8243-d899-406c-aa13-b68157d6b9a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:29.307401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b8243-d899-406c-aa13-b68157d6b9a0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:29.312982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 259/524] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:29 2022] GET /identity/v3/role_assignments?user.id=2cb245e4300e4d5fbdd425e936f4ac77&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:29.313817 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b8243-d899-406c-aa13-b68157d6b9a0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:29.338332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:29.341169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e71f5-dea3-4ddc-8251-c1cbaf94955e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:29.341920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e71f5-dea3-4ddc-8251-c1cbaf94955e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:29.342280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e71f5-dea3-4ddc-8251-c1cbaf94955e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:29.342631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e71f5-dea3-4ddc-8251-c1cbaf94955e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:29.439039 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e71f5-dea3-4ddc-8251-c1cbaf94955e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:29.441982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e71f5-dea3-4ddc-8251-c1cbaf94955e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:29.443654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b8243-d899-406c-aa13-b68157d6b9a0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:29.467900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:29.502893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ec3f2-4a9a-4434-80c5-2a8a02ea6b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:29.503857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ec3f2-4a9a-4434-80c5-2a8a02ea6b1f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:29.504312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ec3f2-4a9a-4434-80c5-2a8a02ea6b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:29.505168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ec3f2-4a9a-4434-80c5-2a8a02ea6b1f None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:29.506385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ec3f2-4a9a-4434-80c5-2a8a02ea6b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:29.506826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ec3f2-4a9a-4434-80c5-2a8a02ea6b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:29.508854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ec3f2-4a9a-4434-80c5-2a8a02ea6b1f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:29.529165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 260/525] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:29.529165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ec3f2-4a9a-4434-80c5-2a8a02ea6b1f None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:29.571345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 266/526] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:29.573340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e71f5-dea3-4ddc-8251-c1cbaf94955e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:29.624363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:29.650246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ec3f2-4a9a-4434-80c5-2a8a02ea6b1f None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:29.684416 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93efdc-bbc1-4391-a446-32fc16e6ed9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:29.685371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93efdc-bbc1-4391-a446-32fc16e6ed9a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:29.685794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93efdc-bbc1-4391-a446-32fc16e6ed9a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:29.686115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e71f5-dea3-4ddc-8251-c1cbaf94955e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:29.686454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93efdc-bbc1-4391-a446-32fc16e6ed9a None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:29.687474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93efdc-bbc1-4391-a446-32fc16e6ed9a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:29.687918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93efdc-bbc1-4391-a446-32fc16e6ed9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:29.689602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93efdc-bbc1-4391-a446-32fc16e6ed9a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:29.709224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:29.725391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d775f06-a43e-4738-8eca-8a51433fb3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:29.726240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d775f06-a43e-4738-8eca-8a51433fb3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:29.726514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d775f06-a43e-4738-8eca-8a51433fb3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:29.726821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d775f06-a43e-4738-8eca-8a51433fb3d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:29.737231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 261/527] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Jul 4 05:41:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:29.737231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93efdc-bbc1-4391-a446-32fc16e6ed9a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:29.738516 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d775f06-a43e-4738-8eca-8a51433fb3d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:29.740029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d775f06-a43e-4738-8eca-8a51433fb3d0 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-5d775f06-a43e-4738-8eca-8a51433fb3d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:29.754772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:29.755687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:29.756953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:29.756953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:29.756953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:29.756953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:29.756953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:29.756953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:29.756953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:29.756953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:29.761014 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 267/528] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:29.761877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d775f06-a43e-4738-8eca-8a51433fb3d0 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:29.865280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93efdc-bbc1-4391-a446-32fc16e6ed9a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:29.878857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d775f06-a43e-4738-8eca-8a51433fb3d0 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:29.905940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:29.913329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93a33e-4173-47ea-a956-d855613e27e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:29.914521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93a33e-4173-47ea-a956-d855613e27e0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:29.914973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93a33e-4173-47ea-a956-d855613e27e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:29.915136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02d49d-5c23-4369-83e0-81020c54ac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:29.915607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93a33e-4173-47ea-a956-d855613e27e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:29.916317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02d49d-5c23-4369-83e0-81020c54ac41 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:29.916824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02d49d-5c23-4369-83e0-81020c54ac41 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:29.917161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02d49d-5c23-4369-83e0-81020c54ac41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:29.928966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93a33e-4173-47ea-a956-d855613e27e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:29.931457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93a33e-4173-47ea-a956-d855613e27e0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-2b93a33e-4173-47ea-a956-d855613e27e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:29.955695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:29.956566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:29.957421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:29.957421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:29.957421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:29.957421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:29.959804 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 262/529] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:29.960892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93a33e-4173-47ea-a956-d855613e27e0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:29.973181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:29.995108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02d49d-5c23-4369-83e0-81020c54ac41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:29.996676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02d49d-5c23-4369-83e0-81020c54ac41 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:30.093157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93a33e-4173-47ea-a956-d855613e27e0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:30.114789 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 268/530] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:30.115002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02d49d-5c23-4369-83e0-81020c54ac41 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:30.145615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808156-cf66-465a-a5cd-9b6de326f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:30.147791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808156-cf66-465a-a5cd-9b6de326f28e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:30.148292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808156-cf66-465a-a5cd-9b6de326f28e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:30.149134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808156-cf66-465a-a5cd-9b6de326f28e None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:30.150477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99808156-cf66-465a-a5cd-9b6de326f28e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:30.150966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99808156-cf66-465a-a5cd-9b6de326f28e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:30.153891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99808156-cf66-465a-a5cd-9b6de326f28e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:30.168704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 263/531] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:30.169890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808156-cf66-465a-a5cd-9b6de326f28e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:30.213503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:30.230551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:30.239982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02d49d-5c23-4369-83e0-81020c54ac41 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:30.287279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93762931-5fcb-4d89-861b-73c6de517459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:30.288502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93762931-5fcb-4d89-861b-73c6de517459 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:30.289085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93762931-5fcb-4d89-861b-73c6de517459 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:30.289424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93762931-5fcb-4d89-861b-73c6de517459 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:30.290213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93762931-5fcb-4d89-861b-73c6de517459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:30.290580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93762931-5fcb-4d89-861b-73c6de517459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:30.291554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808156-cf66-465a-a5cd-9b6de326f28e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:30.292298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93762931-5fcb-4d89-861b-73c6de517459 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:30.306544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 269/532] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:30.307937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93762931-5fcb-4d89-861b-73c6de517459 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:30.340380 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41edb07-0c45-482b-9cbc-27465ef72bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:30.341212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41edb07-0c45-482b-9cbc-27465ef72bde None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:30.341490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41edb07-0c45-482b-9cbc-27465ef72bde None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:30.341763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41edb07-0c45-482b-9cbc-27465ef72bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:30.353107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41edb07-0c45-482b-9cbc-27465ef72bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:30.355002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41edb07-0c45-482b-9cbc-27465ef72bde None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-a41edb07-0c45-482b-9cbc-27465ef72bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:30.375192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:30.376108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:30.376953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:30.376953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:30.376953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:30.376953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:30.383432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 264/533] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:30.384570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41edb07-0c45-482b-9cbc-27465ef72bde None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:30.422309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93762931-5fcb-4d89-861b-73c6de517459 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:30.458690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c770c-d6f8-48e9-9679-5101d75e1809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:30.459398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c770c-d6f8-48e9-9679-5101d75e1809 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:30.459677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c770c-d6f8-48e9-9679-5101d75e1809 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:30.459951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c770c-d6f8-48e9-9679-5101d75e1809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:30.460804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c770c-d6f8-48e9-9679-5101d75e1809 None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=ab33473d40094ec681e538cde250ec2a) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:30.461166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c770c-d6f8-48e9-9679-5101d75e1809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:30.462993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c770c-d6f8-48e9-9679-5101d75e1809 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:30.479447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 270/534] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:30 2022] GET /identity/v3/role_assignments?user.id=ab33473d40094ec681e538cde250ec2a&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:30.479447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c770c-d6f8-48e9-9679-5101d75e1809 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:30.481634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:30.496328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:30.503514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41edb07-0c45-482b-9cbc-27465ef72bde None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:30.549712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2b2f2-ba40-4824-9014-eb0d870238c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:30.550955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b2f2-ba40-4824-9014-eb0d870238c5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:30.551460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b2f2-ba40-4824-9014-eb0d870238c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:30.552006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b2f2-ba40-4824-9014-eb0d870238c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:30.560654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2b2f2-ba40-4824-9014-eb0d870238c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:30.562489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2b2f2-ba40-4824-9014-eb0d870238c5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-0af2b2f2-ba40-4824-9014-eb0d870238c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:30.579630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:30.580542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:30.581371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:30.581371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:30.581371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:30.581371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:30.581371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:30.581371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:30.581371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:30.581371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:30.584193 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 265/535] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:30.585901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2941f74-6856-4083-ae6b-e0696c61c778 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:30.586372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2941f74-6856-4083-ae6b-e0696c61c778 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:30.586796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2941f74-6856-4083-ae6b-e0696c61c778 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:30.589258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 266/536] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:30.590398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2941f74-6856-4083-ae6b-e0696c61c778 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:30.630526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c770c-d6f8-48e9-9679-5101d75e1809 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:30.677748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ad721-2248-45cd-8a07-ce5b5e956976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:30.678570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ad721-2248-45cd-8a07-ce5b5e956976 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:30.678920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ad721-2248-45cd-8a07-ce5b5e956976 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:30.679588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ad721-2248-45cd-8a07-ce5b5e956976 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:30.681308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ad721-2248-45cd-8a07-ce5b5e956976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:30.681735 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ad721-2248-45cd-8a07-ce5b5e956976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:30.683261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ad721-2248-45cd-8a07-ce5b5e956976 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:30.708444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 271/537] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:30.710129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6dfd-c051-4096-a967-bb478cda29db None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:30.710610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6dfd-c051-4096-a967-bb478cda29db None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:30.711105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6dfd-c051-4096-a967-bb478cda29db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:30.718757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2941f74-6856-4083-ae6b-e0696c61c778 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:30.727950 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-422e6dfd-c051-4096-a967-bb478cda29db None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:30.729513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-422e6dfd-c051-4096-a967-bb478cda29db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:30.762812 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8b696-deb6-4562-ae49-0d75bc76d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:30.764010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8b696-deb6-4562-ae49-0d75bc76d4af None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:30.764296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8b696-deb6-4562-ae49-0d75bc76d4af None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:30.764670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8b696-deb6-4562-ae49-0d75bc76d4af None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:30.765472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8b696-deb6-4562-ae49-0d75bc76d4af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:30.765781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8b696-deb6-4562-ae49-0d75bc76d4af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:30.767518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8b696-deb6-4562-ae49-0d75bc76d4af None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:30.789322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 267/538] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:30.790192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8b696-deb6-4562-ae49-0d75bc76d4af None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:30.809549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:30.824031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:30.832033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 272/539] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:30.833953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e6dfd-c051-4096-a967-bb478cda29db None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:30.929658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8b696-deb6-4562-ae49-0d75bc76d4af None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:30.967317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e6dfd-c051-4096-a967-bb478cda29db None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:30.984482 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6c151-7144-48c0-8794-51702e76f9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:30.985214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6c151-7144-48c0-8794-51702e76f9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:30.985490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6c151-7144-48c0-8794-51702e76f9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:30.985757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6c151-7144-48c0-8794-51702e76f9d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:30.995498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6c151-7144-48c0-8794-51702e76f9d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:30.997465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6c151-7144-48c0-8794-51702e76f9d1 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-2bc6c151-7144-48c0-8794-51702e76f9d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:31.017157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:31.018053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:31.019346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:31.019346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:31.019346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:31.019346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:31.019346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:31.019346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:31.019346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:31.019346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:31.019346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 268/540] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:31.019346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6c151-7144-48c0-8794-51702e76f9d1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:31.021277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96c443-2635-4254-971e-b91e59c6601d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:31.022480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96c443-2635-4254-971e-b91e59c6601d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:31.023305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96c443-2635-4254-971e-b91e59c6601d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:31.024875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96c443-2635-4254-971e-b91e59c6601d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:31.026097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96c443-2635-4254-971e-b91e59c6601d None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=0f63e110260e4471b0a54576d3573583) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:31.026584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96c443-2635-4254-971e-b91e59c6601d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:31.028276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96c443-2635-4254-971e-b91e59c6601d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:31.037216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 273/541] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:30 2022] GET /identity/v3/role_assignments?user.id=0f63e110260e4471b0a54576d3573583&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:31.038443 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96c443-2635-4254-971e-b91e59c6601d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:31.113795 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:31.169059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96c443-2635-4254-971e-b91e59c6601d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:31.196649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:31.223237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6c151-7144-48c0-8794-51702e76f9d1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:31.224426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e56c5-fb74-4b01-a125-84e1813b7148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:31.225763 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e56c5-fb74-4b01-a125-84e1813b7148 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:31.227332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e56c5-fb74-4b01-a125-84e1813b7148 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:31.227838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e56c5-fb74-4b01-a125-84e1813b7148 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:31.229200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e56c5-fb74-4b01-a125-84e1813b7148 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:31.229768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e56c5-fb74-4b01-a125-84e1813b7148 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:31.231875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e56c5-fb74-4b01-a125-84e1813b7148 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:31.257610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 274/542] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:31.266739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e56c5-fb74-4b01-a125-84e1813b7148 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:31.290719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94b6e-a777-4ffb-86ee-949f8b39253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:31.291961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94b6e-a777-4ffb-86ee-949f8b39253a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:31.292431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94b6e-a777-4ffb-86ee-949f8b39253a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:31.292972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94b6e-a777-4ffb-86ee-949f8b39253a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:31.313562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a94b6e-a777-4ffb-86ee-949f8b39253a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-75a94b6e-a777-4ffb-86ee-949f8b39253a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:31.327587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:31.328530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:31.329307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 269/543] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:31.338582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94b6e-a777-4ffb-86ee-949f8b39253a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:31.493926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e56c5-fb74-4b01-a125-84e1813b7148 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:31.547642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:31.574186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949923e-5b42-4845-8ad3-b9fe178ebb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:31.574595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94b6e-a777-4ffb-86ee-949f8b39253a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:31.575031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949923e-5b42-4845-8ad3-b9fe178ebb28 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:31.575106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949923e-5b42-4845-8ad3-b9fe178ebb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:31.575412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949923e-5b42-4845-8ad3-b9fe178ebb28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:31.576259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949923e-5b42-4845-8ad3-b9fe178ebb28 None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=817b1a026f4b443daea514338ad8531a) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:31.576572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949923e-5b42-4845-8ad3-b9fe178ebb28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:31.578040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949923e-5b42-4845-8ad3-b9fe178ebb28 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:31.584424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 275/544] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:31 2022] GET /identity/v3/role_assignments?user.id=817b1a026f4b443daea514338ad8531a&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:31.585745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3c0e1-f164-4d7b-ab9c-55c44723416e None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:31.586032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3c0e1-f164-4d7b-ab9c-55c44723416e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:31.586308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3c0e1-f164-4d7b-ab9c-55c44723416e None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:31.592039 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 276/545] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:31.601098 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a2e8-f086-4e5c-8b92-9bdea0f5f8f1 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:31.601419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a2e8-f086-4e5c-8b92-9bdea0f5f8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:31.601691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a2e8-f086-4e5c-8b92-9bdea0f5f8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:31.637134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b704a2e8-f086-4e5c-8b92-9bdea0f5f8f1 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:31.638375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b704a2e8-f086-4e5c-8b92-9bdea0f5f8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:31.673278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d639479-1104-4438-9e68-5c050e729d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:31.673980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639479-1104-4438-9e68-5c050e729d66 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:31.674262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639479-1104-4438-9e68-5c050e729d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:31.674575 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639479-1104-4438-9e68-5c050e729d66 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:31.675687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d639479-1104-4438-9e68-5c050e729d66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:31.675994 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d639479-1104-4438-9e68-5c050e729d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:31.677541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d639479-1104-4438-9e68-5c050e729d66 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:31.682816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:31.698298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 270/546] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:31.704121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d639479-1104-4438-9e68-5c050e729d66 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:31.764983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 277/547] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:31.786028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a2e8-f086-4e5c-8b92-9bdea0f5f8f1 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:31.904873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d639479-1104-4438-9e68-5c050e729d66 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:31.921431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a2e8-f086-4e5c-8b92-9bdea0f5f8f1 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:31.947609 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:31.947978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbb27a-6461-4816-a9b2-a717ac8f47b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:31.948795 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbb27a-6461-4816-a9b2-a717ac8f47b3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:31.949085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbb27a-6461-4816-a9b2-a717ac8f47b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:31.949331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbb27a-6461-4816-a9b2-a717ac8f47b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:31.968262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbb27a-6461-4816-a9b2-a717ac8f47b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:31.970953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbb27a-6461-4816-a9b2-a717ac8f47b3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:31.993427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d7e8d-3e4d-48c7-a939-35433599f179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:31.994386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d7e8d-3e4d-48c7-a939-35433599f179 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:31.994662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d7e8d-3e4d-48c7-a939-35433599f179 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:31.994929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d7e8d-3e4d-48c7-a939-35433599f179 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-25bbb27a-6461-4816-a9b2-a717ac8f47b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:32.005640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:32.006505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:32.007380 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:32.007380 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:32.007380 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:32.007380 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:32.008255 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 271/548] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:32.013173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbb27a-6461-4816-a9b2-a717ac8f47b3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:32.014624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d7e8d-3e4d-48c7-a939-35433599f179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-1d3d7e8d-3e4d-48c7-a939-35433599f179 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:32.033846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:32.034844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:32.037120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 278/549] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:32.049111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d7e8d-3e4d-48c7-a939-35433599f179 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:32.076748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:32.132183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbb27a-6461-4816-a9b2-a717ac8f47b3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:32.167441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cb3ca-863c-49b0-ad0c-4ce6de6d5981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:32.168115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cb3ca-863c-49b0-ad0c-4ce6de6d5981 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:32.168389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cb3ca-863c-49b0-ad0c-4ce6de6d5981 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:32.168659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cb3ca-863c-49b0-ad0c-4ce6de6d5981 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:32.169385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cb3ca-863c-49b0-ad0c-4ce6de6d5981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:32.169688 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cb3ca-863c-49b0-ad0c-4ce6de6d5981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:32.171057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cb3ca-863c-49b0-ad0c-4ce6de6d5981 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:32.183793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 272/550] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:32.188709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cb3ca-863c-49b0-ad0c-4ce6de6d5981 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:32.239209 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d7e8d-3e4d-48c7-a939-35433599f179 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:32.295293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cb3ca-863c-49b0-ad0c-4ce6de6d5981 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:32.300889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9f7d-1019-405c-b40b-ead477f24d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:32.301585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9f7d-1019-405c-b40b-ead477f24d4f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:32.301866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9f7d-1019-405c-b40b-ead477f24d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:32.302138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9f7d-1019-405c-b40b-ead477f24d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:32.302902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9f7d-1019-405c-b40b-ead477f24d4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:32.303207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9f7d-1019-405c-b40b-ead477f24d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:32.304655 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9f7d-1019-405c-b40b-ead477f24d4f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:32.306296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:32.323490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 279/551] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:32.329375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9f7d-1019-405c-b40b-ead477f24d4f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:32.347571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affabea1-9af5-42af-b0e8-76c7221ddad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:32.349687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affabea1-9af5-42af-b0e8-76c7221ddad9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:32.349984 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affabea1-9af5-42af-b0e8-76c7221ddad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:32.350446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affabea1-9af5-42af-b0e8-76c7221ddad9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:32.368828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affabea1-9af5-42af-b0e8-76c7221ddad9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:32.370465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affabea1-9af5-42af-b0e8-76c7221ddad9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:32.397027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-affabea1-9af5-42af-b0e8-76c7221ddad9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:32.405859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:32.408111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:32.410402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 273/552] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:32.416131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affabea1-9af5-42af-b0e8-76c7221ddad9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:32.549449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9f7d-1019-405c-b40b-ead477f24d4f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:32.559013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affabea1-9af5-42af-b0e8-76c7221ddad9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:32.594364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e7876-0e1d-42fe-82fe-2394fcf2b5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:32.595153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e7876-0e1d-42fe-82fe-2394fcf2b5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:32.595355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e7876-0e1d-42fe-82fe-2394fcf2b5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:32.595627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e7876-0e1d-42fe-82fe-2394fcf2b5c1 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:32.596464 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e7876-0e1d-42fe-82fe-2394fcf2b5c1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:32.596761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e7876-0e1d-42fe-82fe-2394fcf2b5c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:32.603271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e7876-0e1d-42fe-82fe-2394fcf2b5c1 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:32.617309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 274/553] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 4 05:41:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:32.622326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e7876-0e1d-42fe-82fe-2394fcf2b5c1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:32.635973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9321a-6230-4f4f-b6a1-0dbd4b60ee9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:32.636777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9321a-6230-4f4f-b6a1-0dbd4b60ee9a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:32.637048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9321a-6230-4f4f-b6a1-0dbd4b60ee9a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:32.637318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9321a-6230-4f4f-b6a1-0dbd4b60ee9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:32.659291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9321a-6230-4f4f-b6a1-0dbd4b60ee9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:32.660778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9321a-6230-4f4f-b6a1-0dbd4b60ee9a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:32.674935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-3ae9321a-6230-4f4f-b6a1-0dbd4b60ee9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:32.688981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:32.690136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:32.691217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:32.691217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:32.691217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:32.691217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:32.694115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 280/554] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:32.706100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9321a-6230-4f4f-b6a1-0dbd4b60ee9a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:32.810886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e7876-0e1d-42fe-82fe-2394fcf2b5c1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:32.857278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:32.874190 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9321a-6230-4f4f-b6a1-0dbd4b60ee9a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:32.876985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691debd-5ff8-4a5a-a446-68a04db50de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:32.877652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691debd-5ff8-4a5a-a446-68a04db50de8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:32.877935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691debd-5ff8-4a5a-a446-68a04db50de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:32.878206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691debd-5ff8-4a5a-a446-68a04db50de8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:32.912141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6010-f2d7-41e2-b87d-7823b0735380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:32.913261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6010-f2d7-41e2-b87d-7823b0735380 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:32.913981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6010-f2d7-41e2-b87d-7823b0735380 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:32.914453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6010-f2d7-41e2-b87d-7823b0735380 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:32.915758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d6010-f2d7-41e2-b87d-7823b0735380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:32.916293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d6010-f2d7-41e2-b87d-7823b0735380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:32.918461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d6010-f2d7-41e2-b87d-7823b0735380 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:32.934613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 281/555] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:32.940761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6010-f2d7-41e2-b87d-7823b0735380 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:33.039572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691debd-5ff8-4a5a-a446-68a04db50de8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:33.041258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691debd-5ff8-4a5a-a446-68a04db50de8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:33.055717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6010-f2d7-41e2-b87d-7823b0735380 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:33.070307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:33.101513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffe090-f8fb-402f-ac96-e89a95944f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:33.102929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe090-f8fb-402f-ac96-e89a95944f44 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.103863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe090-f8fb-402f-ac96-e89a95944f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.104709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe090-f8fb-402f-ac96-e89a95944f44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.114920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffe090-f8fb-402f-ac96-e89a95944f44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:33.116932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffe090-f8fb-402f-ac96-e89a95944f44 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-afffe090-f8fb-402f-ac96-e89a95944f44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:33.138181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:33.139680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:33.140621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:33.140621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:33.140621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:33.140621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:33.140621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:33.140621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:33.140621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:33.140621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:33.140621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:33.140621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:33.141154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 282/556] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 4 05:41:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:33.147009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45409b-ee9f-498c-ba32-694ac1c5da79 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.147009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45409b-ee9f-498c-ba32-694ac1c5da79 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.147009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45409b-ee9f-498c-ba32-694ac1c5da79 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.149073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 283/557] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:33.151917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45409b-ee9f-498c-ba32-694ac1c5da79 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:33.156302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 275/558] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 05:41:33.158231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc617c-b853-408a-83ac-9858232b6a95 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.159765 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc617c-b853-408a-83ac-9858232b6a95 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.160259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc617c-b853-408a-83ac-9858232b6a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.165147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:33.183348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-51cc617c-b853-408a-83ac-9858232b6a95 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:33.185362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-51cc617c-b853-408a-83ac-9858232b6a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:33.273312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 276/559] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:33.274631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc617c-b853-408a-83ac-9858232b6a95 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:33.294812 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45409b-ee9f-498c-ba32-694ac1c5da79 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:33.337878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19609684-b2ca-4ce2-98b8-4cf56c9e1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:33.339045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19609684-b2ca-4ce2-98b8-4cf56c9e1a46 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.339553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19609684-b2ca-4ce2-98b8-4cf56c9e1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.340027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19609684-b2ca-4ce2-98b8-4cf56c9e1a46 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.341070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19609684-b2ca-4ce2-98b8-4cf56c9e1a46 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:33.341672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19609684-b2ca-4ce2-98b8-4cf56c9e1a46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:33.343467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19609684-b2ca-4ce2-98b8-4cf56c9e1a46 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:33.348013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:33.368954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 284/560] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 4 05:41:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:33.370364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19609684-b2ca-4ce2-98b8-4cf56c9e1a46 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:33.390834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc617c-b853-408a-83ac-9858232b6a95 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:33.426697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216272f-a587-4c87-b53c-036c72b39744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:33.427795 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216272f-a587-4c87-b53c-036c72b39744 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.428075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216272f-a587-4c87-b53c-036c72b39744 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.428591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216272f-a587-4c87-b53c-036c72b39744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.435688 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:33.444541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216272f-a587-4c87-b53c-036c72b39744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:33.446256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216272f-a587-4c87-b53c-036c72b39744 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-e216272f-a587-4c87-b53c-036c72b39744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:33.465234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:33.466268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:33.467362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:33.467362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:33.467362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:33.467362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:33.471038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 277/561] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:33.472003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216272f-a587-4c87-b53c-036c72b39744 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:33.477831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19609684-b2ca-4ce2-98b8-4cf56c9e1a46 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:33.518331 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7130bf4-b0b7-49f4-8aa9-4d7d9a5fcba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:33.519141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7130bf4-b0b7-49f4-8aa9-4d7d9a5fcba0 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.519452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7130bf4-b0b7-49f4-8aa9-4d7d9a5fcba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.519729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7130bf4-b0b7-49f4-8aa9-4d7d9a5fcba0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.530911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7130bf4-b0b7-49f4-8aa9-4d7d9a5fcba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-a7130bf4-b0b7-49f4-8aa9-4d7d9a5fcba0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:33.539333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:33.540352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:33.541257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 285/562] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:33.542421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424ff2e-53a3-4e42-bf77-17850a0df179 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.542421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424ff2e-53a3-4e42-bf77-17850a0df179 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.542579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424ff2e-53a3-4e42-bf77-17850a0df179 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.543888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 286/563] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:33.544466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424ff2e-53a3-4e42-bf77-17850a0df179 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:33.588334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216272f-a587-4c87-b53c-036c72b39744 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:33.600235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:33.625981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e5586-2ab6-44cf-b6c7-0c0b63b04b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:33.626896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5586-2ab6-44cf-b6c7-0c0b63b04b22 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.627067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5586-2ab6-44cf-b6c7-0c0b63b04b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.627379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5586-2ab6-44cf-b6c7-0c0b63b04b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.639644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e5586-2ab6-44cf-b6c7-0c0b63b04b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:33.641119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e5586-2ab6-44cf-b6c7-0c0b63b04b22 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:33.657073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424ff2e-53a3-4e42-bf77-17850a0df179 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-b18e5586-2ab6-44cf-b6c7-0c0b63b04b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:33.660846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:33.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:33.667402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:33.667402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:33.667402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:33.667402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:33.667402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 278/564] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:33.667402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e971780-4ff1-4d09-860a-27c310526486 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.667402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e971780-4ff1-4d09-860a-27c310526486 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.667402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e971780-4ff1-4d09-860a-27c310526486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.681021 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-0e971780-4ff1-4d09-860a-27c310526486 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:33.682334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0e971780-4ff1-4d09-860a-27c310526486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:33.685428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:33.697531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed235a-a867-4e48-aaf3-3ad8af947878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:33.698252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed235a-a867-4e48-aaf3-3ad8af947878 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.698538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed235a-a867-4e48-aaf3-3ad8af947878 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.698841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed235a-a867-4e48-aaf3-3ad8af947878 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.699595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed235a-a867-4e48-aaf3-3ad8af947878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:33.699898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed235a-a867-4e48-aaf3-3ad8af947878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:33.701278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed235a-a867-4e48-aaf3-3ad8af947878 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:33.713831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 287/565] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:33.714561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed235a-a867-4e48-aaf3-3ad8af947878 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:33.757616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 279/566] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:33.758649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e971780-4ff1-4d09-860a-27c310526486 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:33.818114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed235a-a867-4e48-aaf3-3ad8af947878 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:33.846467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:33.852031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842c9a-e908-4145-be96-ab75777b55e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:33.852709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842c9a-e908-4145-be96-ab75777b55e9 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.852986 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842c9a-e908-4145-be96-ab75777b55e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.853293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842c9a-e908-4145-be96-ab75777b55e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.854043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27842c9a-e908-4145-be96-ab75777b55e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:33.854342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27842c9a-e908-4145-be96-ab75777b55e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:33.855146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e971780-4ff1-4d09-860a-27c310526486 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:33.855796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27842c9a-e908-4145-be96-ab75777b55e9 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:33.863799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 288/567] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:33.864589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842c9a-e908-4145-be96-ab75777b55e9 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:33.892896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31353fc8-5cb7-4154-b178-f673e8a7e1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:33.893602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31353fc8-5cb7-4154-b178-f673e8a7e1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.893882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31353fc8-5cb7-4154-b178-f673e8a7e1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.894156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31353fc8-5cb7-4154-b178-f673e8a7e1c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.894881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31353fc8-5cb7-4154-b178-f673e8a7e1c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:33.895185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31353fc8-5cb7-4154-b178-f673e8a7e1c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:33.896594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31353fc8-5cb7-4154-b178-f673e8a7e1c5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:33.907889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 280/568] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:33.909147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba06eb9-d81c-4636-b98c-374106fc2ed0 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.909440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba06eb9-d81c-4636-b98c-374106fc2ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:33.909791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba06eb9-d81c-4636-b98c-374106fc2ed0 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:33.913182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:33.915462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 281/569] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:33.916392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba06eb9-d81c-4636-b98c-374106fc2ed0 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:33.963991 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842c9a-e908-4145-be96-ab75777b55e9 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:33.998634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc057f-0bf0-49f9-9f52-461312864145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:33.999465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc057f-0bf0-49f9-9f52-461312864145 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:33.999762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc057f-0bf0-49f9-9f52-461312864145 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.000031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc057f-0bf0-49f9-9f52-461312864145 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.006579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc057f-0bf0-49f9-9f52-461312864145 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:34.008088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc057f-0bf0-49f9-9f52-461312864145 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:34.015909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba06eb9-d81c-4636-b98c-374106fc2ed0 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-a4dc057f-0bf0-49f9-9f52-461312864145 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:34.023045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:34.024000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:34.024886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:34.024886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:34.024886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:34.024886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:34.024886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:34.024886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:34.024886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:34.024886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:34.024886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 289/570] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:34.025445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc057f-0bf0-49f9-9f52-461312864145 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.055171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e683cce-1e0d-409f-a2a7-6862da515543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.056082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e683cce-1e0d-409f-a2a7-6862da515543 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.056445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e683cce-1e0d-409f-a2a7-6862da515543 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.056721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e683cce-1e0d-409f-a2a7-6862da515543 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.066669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e683cce-1e0d-409f-a2a7-6862da515543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-1e683cce-1e0d-409f-a2a7-6862da515543 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:34.074838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:34.075781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:34.076615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 282/571] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:34.077308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e683cce-1e0d-409f-a2a7-6862da515543 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.091218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:34.136157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc057f-0bf0-49f9-9f52-461312864145 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.144384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:34.173493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acba8b3-a616-4b43-b351-e6f17d31be58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.174299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acba8b3-a616-4b43-b351-e6f17d31be58 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.174579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acba8b3-a616-4b43-b351-e6f17d31be58 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.174851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acba8b3-a616-4b43-b351-e6f17d31be58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.178709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e683cce-1e0d-409f-a2a7-6862da515543 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.185257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acba8b3-a616-4b43-b351-e6f17d31be58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:34.186742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acba8b3-a616-4b43-b351-e6f17d31be58 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-3acba8b3-a616-4b43-b351-e6f17d31be58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:34.205143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:34.206106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:34.206978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:34.206978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:34.206978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:34.206978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:34.206978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 290/572] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:34.207729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed2b5-3531-407e-8b9e-6732ef7ca98b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.208008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed2b5-3531-407e-8b9e-6732ef7ca98b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.208280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed2b5-3531-407e-8b9e-6732ef7ca98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.216542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92135514-7068-4ae0-bcc9-b2fce919e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.217341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92135514-7068-4ae0-bcc9-b2fce919e39b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.217615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92135514-7068-4ae0-bcc9-b2fce919e39b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.217905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92135514-7068-4ae0-bcc9-b2fce919e39b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.224624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92135514-7068-4ae0-bcc9-b2fce919e39b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:34.224762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-9eaed2b5-3531-407e-8b9e-6732ef7ca98b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:34.225944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9eaed2b5-3531-407e-8b9e-6732ef7ca98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:34.226037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92135514-7068-4ae0-bcc9-b2fce919e39b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-92135514-7068-4ae0-bcc9-b2fce919e39b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:34.240314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:34.241264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:34.242134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:34.242134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:34.242134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:34.242134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:34.242134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:34.242134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:34.242134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:34.242134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:34.246398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 283/573] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:34.247279 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92135514-7068-4ae0-bcc9-b2fce919e39b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.292615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 291/574] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:34.293472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaed2b5-3531-407e-8b9e-6732ef7ca98b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.317437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:34.348707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92135514-7068-4ae0-bcc9-b2fce919e39b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.381365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b188fe-5d7d-4211-8a6a-d875b933394b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.381561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:34.382025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b188fe-5d7d-4211-8a6a-d875b933394b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.382304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b188fe-5d7d-4211-8a6a-d875b933394b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.382580 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b188fe-5d7d-4211-8a6a-d875b933394b None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.383391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b188fe-5d7d-4211-8a6a-d875b933394b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:34.383698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b188fe-5d7d-4211-8a6a-d875b933394b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:34.385221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b188fe-5d7d-4211-8a6a-d875b933394b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:34.394110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaed2b5-3531-407e-8b9e-6732ef7ca98b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.396541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 284/575] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:34.397327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b188fe-5d7d-4211-8a6a-d875b933394b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.427321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8cded-ea88-4b7a-9496-9f84bcfa0304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.428022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8cded-ea88-4b7a-9496-9f84bcfa0304 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.428297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8cded-ea88-4b7a-9496-9f84bcfa0304 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.428565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8cded-ea88-4b7a-9496-9f84bcfa0304 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.429326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8cded-ea88-4b7a-9496-9f84bcfa0304 None admin] RBAC: Adding query filter params (name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:34.429631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8cded-ea88-4b7a-9496-9f84bcfa0304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:34.431007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8cded-ea88-4b7a-9496-9f84bcfa0304 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:34.443800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 292/576] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:34.443800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8cded-ea88-4b7a-9496-9f84bcfa0304 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.500614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b188fe-5d7d-4211-8a6a-d875b933394b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.531411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:34.537345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650988c-e8a5-4df8-95a9-7bca42ef3f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.538009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650988c-e8a5-4df8-95a9-7bca42ef3f01 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.538343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650988c-e8a5-4df8-95a9-7bca42ef3f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.538611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650988c-e8a5-4df8-95a9-7bca42ef3f01 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.539372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650988c-e8a5-4df8-95a9-7bca42ef3f01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:34.539674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650988c-e8a5-4df8-95a9-7bca42ef3f01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:34.544087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650988c-e8a5-4df8-95a9-7bca42ef3f01 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:34.550006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8cded-ea88-4b7a-9496-9f84bcfa0304 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.553748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 285/577] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:34.554589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650988c-e8a5-4df8-95a9-7bca42ef3f01 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.585892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0255c07-92f2-4505-9c21-e1c355ba9ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.586546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0255c07-92f2-4505-9c21-e1c355ba9ceb None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.586824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0255c07-92f2-4505-9c21-e1c355ba9ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.587100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0255c07-92f2-4505-9c21-e1c355ba9ceb None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.587953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0255c07-92f2-4505-9c21-e1c355ba9ceb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:34.588232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0255c07-92f2-4505-9c21-e1c355ba9ceb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:34.589688 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0255c07-92f2-4505-9c21-e1c355ba9ceb None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:34.601842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 293/578] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:34.602627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0255c07-92f2-4505-9c21-e1c355ba9ceb None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.618184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:34.655120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650988c-e8a5-4df8-95a9-7bca42ef3f01 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.687411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53805a82-e7ce-42c7-9b3f-a01d405987c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.688220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805a82-e7ce-42c7-9b3f-a01d405987c1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.688498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805a82-e7ce-42c7-9b3f-a01d405987c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.688769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805a82-e7ce-42c7-9b3f-a01d405987c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.699153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53805a82-e7ce-42c7-9b3f-a01d405987c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:34.705605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0255c07-92f2-4505-9c21-e1c355ba9ceb None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-53805a82-e7ce-42c7-9b3f-a01d405987c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:34.706938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:34.707975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:34.708810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 286/579] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:34.709506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53805a82-e7ce-42c7-9b3f-a01d405987c1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.744706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ea1ea-147c-4623-a76e-b3c0734b994c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.745368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ea1ea-147c-4623-a76e-b3c0734b994c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.745645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ea1ea-147c-4623-a76e-b3c0734b994c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.745918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ea1ea-147c-4623-a76e-b3c0734b994c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.746755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ea1ea-147c-4623-a76e-b3c0734b994c None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=655b3c5b98fa4d91b328d04d42fc263a) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:34.747107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ea1ea-147c-4623-a76e-b3c0734b994c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:34.748561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ea1ea-147c-4623-a76e-b3c0734b994c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:34.754029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 294/580] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/role_assignments?user.id=655b3c5b98fa4d91b328d04d42fc263a&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:34.754640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ea1ea-147c-4623-a76e-b3c0734b994c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.766327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:34.826070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53805a82-e7ce-42c7-9b3f-a01d405987c1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.844379 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:34.865766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15d37c-6c53-488c-a005-440256ceb261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.866586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15d37c-6c53-488c-a005-440256ceb261 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.866866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15d37c-6c53-488c-a005-440256ceb261 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.867135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15d37c-6c53-488c-a005-440256ceb261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.872699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ea1ea-147c-4623-a76e-b3c0734b994c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:34.879539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15d37c-6c53-488c-a005-440256ceb261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:34.881086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15d37c-6c53-488c-a005-440256ceb261 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-7a15d37c-6c53-488c-a005-440256ceb261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:34.899813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:34.901089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:34.902049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:34.902049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:34.902049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:34.902049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:34.902049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 287/581] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:34.902049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15d37c-6c53-488c-a005-440256ceb261 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:34.913668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b9e9d-d11f-4f63-9841-27d97db22f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:34.914398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b9e9d-d11f-4f63-9841-27d97db22f36 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:34.914679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b9e9d-d11f-4f63-9841-27d97db22f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:34.914953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b9e9d-d11f-4f63-9841-27d97db22f36 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:34.922710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b9e9d-d11f-4f63-9841-27d97db22f36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:34.924188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b9e9d-d11f-4f63-9841-27d97db22f36 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-976b9e9d-d11f-4f63-9841-27d97db22f36 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:34.939578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:34.940560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:34.941478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 295/582] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:34.942474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b9e9d-d11f-4f63-9841-27d97db22f36 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.017637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15d37c-6c53-488c-a005-440256ceb261 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:35.019930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:35.044218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b9e9d-d11f-4f63-9841-27d97db22f36 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:35.053699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1a4f9-2421-43b8-88a0-7cd51a250696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:35.054608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1a4f9-2421-43b8-88a0-7cd51a250696 None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:35.054886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1a4f9-2421-43b8-88a0-7cd51a250696 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:35.055155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1a4f9-2421-43b8-88a0-7cd51a250696 None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/2cb245e4300e4d5fbdd425e936f4ac77/roles/04f8b72053764c7f8067d4b478d31e95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:35.072923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1a4f9-2421-43b8-88a0-7cd51a250696 None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=2cb245e4300e4d5fbdd425e936f4ac77, role_id=04f8b72053764c7f8067d4b478d31e95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:35.074373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1a4f9-2421-43b8-88a0-7cd51a250696 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:35.081744 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1702f5b-2ae1-4323-b887-c49ff0a96ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:35.082448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1702f5b-2ae1-4323-b887-c49ff0a96ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:35.082787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1702f5b-2ae1-4323-b887-c49ff0a96ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:35.083007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1702f5b-2ae1-4323-b887-c49ff0a96ec3 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:35.083845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1702f5b-2ae1-4323-b887-c49ff0a96ec3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:35.084155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1702f5b-2ae1-4323-b887-c49ff0a96ec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:35.085477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1702f5b-2ae1-4323-b887-c49ff0a96ec3 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:35.086937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:35.089434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 288/583] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:41:34 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/2cb245e4300e4d5fbdd425e936f4ac77/roles/04f8b72053764c7f8067d4b478d31e95 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:35.090239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1a4f9-2421-43b8-88a0-7cd51a250696 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.099972 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 296/584] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:35.100984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1702f5b-2ae1-4323-b887-c49ff0a96ec3 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.234778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1702f5b-2ae1-4323-b887-c49ff0a96ec3 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:35.247748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1a4f9-2421-43b8-88a0-7cd51a250696 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:35.274584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c752d-e397-4e78-a62b-4a7df3e40ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:35.275532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c752d-e397-4e78-a62b-4a7df3e40ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:35.275928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c752d-e397-4e78-a62b-4a7df3e40ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:35.276747 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c752d-e397-4e78-a62b-4a7df3e40ff4 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:35.277076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c752d-e397-4e78-a62b-4a7df3e40ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:35.277382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c752d-e397-4e78-a62b-4a7df3e40ff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:35.278833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c752d-e397-4e78-a62b-4a7df3e40ff4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:35.293821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fb739-abb4-45e0-b59b-618894138b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:35.295064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fb739-abb4-45e0-b59b-618894138b41 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:35.295376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fb739-abb4-45e0-b59b-618894138b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:35.295652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fb739-abb4-45e0-b59b-618894138b41 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:35.296676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fb739-abb4-45e0-b59b-618894138b41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:35.296984 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fb739-abb4-45e0-b59b-618894138b41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:35.298396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fb739-abb4-45e0-b59b-618894138b41 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:35.299182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 297/585] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 4 05:41:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:35.300541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c752d-e397-4e78-a62b-4a7df3e40ff4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.312677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 289/586] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:35.313304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fb739-abb4-45e0-b59b-618894138b41 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.336246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:35.370233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:35.428208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fb739-abb4-45e0-b59b-618894138b41 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:35.431568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c752d-e397-4e78-a62b-4a7df3e40ff4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:35.464928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5621e-7bcb-4769-acc0-537d6a1a60bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:35.466050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5621e-7bcb-4769-acc0-537d6a1a60bf None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:35.466363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5621e-7bcb-4769-acc0-537d6a1a60bf None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:35.466638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5621e-7bcb-4769-acc0-537d6a1a60bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:35.474585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533efe3-27f3-4aa6-ab12-cef152a74af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:35.475718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533efe3-27f3-4aa6-ab12-cef152a74af5 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:35.476547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533efe3-27f3-4aa6-ab12-cef152a74af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:35.476999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533efe3-27f3-4aa6-ab12-cef152a74af5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:35.478247 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5621e-7bcb-4769-acc0-537d6a1a60bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:35.480086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5621e-7bcb-4769-acc0-537d6a1a60bf None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:35.489985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533efe3-27f3-4aa6-ab12-cef152a74af5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:35.491946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533efe3-27f3-4aa6-ab12-cef152a74af5 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-afd5621e-7bcb-4769-acc0-537d6a1a60bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:35.502080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:35.503353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:35.504452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:35.504452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:35.504452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:35.504452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:35.504452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 290/587] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:35.504818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5621e-7bcb-4769-acc0-537d6a1a60bf None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-2533efe3-27f3-4aa6-ab12-cef152a74af5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:35.518960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:35.520003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:35.520897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:35.520897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:35.520897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:35.520897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:35.527369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 298/588] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:35.528684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533efe3-27f3-4aa6-ab12-cef152a74af5 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.597224 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:35.641143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5621e-7bcb-4769-acc0-537d6a1a60bf None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:35.657371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533efe3-27f3-4aa6-ab12-cef152a74af5 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:35.666945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:35.682072 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20fc31-efd9-4831-81d4-b60ef17dce36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:35.682889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20fc31-efd9-4831-81d4-b60ef17dce36 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:35.683171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20fc31-efd9-4831-81d4-b60ef17dce36 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:35.683490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20fc31-efd9-4831-81d4-b60ef17dce36 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:35.693542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20fc31-efd9-4831-81d4-b60ef17dce36 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:35.695819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20fc31-efd9-4831-81d4-b60ef17dce36 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:35.702214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9ac6d-99f1-4852-9156-d94cb7c0c5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:35.703378 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9ac6d-99f1-4852-9156-d94cb7c0c5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:35.703829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9ac6d-99f1-4852-9156-d94cb7c0c5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:35.704292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9ac6d-99f1-4852-9156-d94cb7c0c5d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:35.705184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9ac6d-99f1-4852-9156-d94cb7c0c5d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:35.705653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9ac6d-99f1-4852-9156-d94cb7c0c5d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:35.707702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9ac6d-99f1-4852-9156-d94cb7c0c5d1 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-0e20fc31-efd9-4831-81d4-b60ef17dce36 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:35.712264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:35.713377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:35.714292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:35.714292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:35.714292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:35.714292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:35.714292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:35.714292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:35.714292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:35.714292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:35.714292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 05:41:35.714292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:35.719122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 291/589] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:35.720059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20fc31-efd9-4831-81d4-b60ef17dce36 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.725547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 299/590] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:35.726595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9ac6d-99f1-4852-9156-d94cb7c0c5d1 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.840144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9ac6d-99f1-4852-9156-d94cb7c0c5d1 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:35.847390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20fc31-efd9-4831-81d4-b60ef17dce36 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:35.881293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d577be-3b3d-4453-8329-e9df883d2812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:35.882540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d577be-3b3d-4453-8329-e9df883d2812 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:35.882998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d577be-3b3d-4453-8329-e9df883d2812 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:35.883566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:35.883782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d577be-3b3d-4453-8329-e9df883d2812 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:35.884957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d577be-3b3d-4453-8329-e9df883d2812 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:35.885836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d577be-3b3d-4453-8329-e9df883d2812 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:35.887871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d577be-3b3d-4453-8329-e9df883d2812 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:35.894873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02b8b2-f5ec-4f31-96a3-e79e29acd1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:35.895660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02b8b2-f5ec-4f31-96a3-e79e29acd1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:35.896012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02b8b2-f5ec-4f31-96a3-e79e29acd1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:35.896358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02b8b2-f5ec-4f31-96a3-e79e29acd1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:35.897157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02b8b2-f5ec-4f31-96a3-e79e29acd1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:35.897548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02b8b2-f5ec-4f31-96a3-e79e29acd1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:35.898981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02b8b2-f5ec-4f31-96a3-e79e29acd1f3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:35.911320 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 300/591] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 4 05:41:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:35.912413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d577be-3b3d-4453-8329-e9df883d2812 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.919403 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 292/592] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:35.919403 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02b8b2-f5ec-4f31-96a3-e79e29acd1f3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:35.947396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:36.021468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02b8b2-f5ec-4f31-96a3-e79e29acd1f3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:36.039204 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d577be-3b3d-4453-8329-e9df883d2812 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:36.065071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b78f-9d42-4506-99e4-6c3ff9ceea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:36.066014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b78f-9d42-4506-99e4-6c3ff9ceea03 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.066395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b78f-9d42-4506-99e4-6c3ff9ceea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.066673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b78f-9d42-4506-99e4-6c3ff9ceea03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.076756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce825b-119b-4a6b-be88-47199ccc54af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:36.077861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce825b-119b-4a6b-be88-47199ccc54af None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.078336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce825b-119b-4a6b-be88-47199ccc54af None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.078817 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce825b-119b-4a6b-be88-47199ccc54af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.079038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651b78f-9d42-4506-99e4-6c3ff9ceea03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:36.080478 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651b78f-9d42-4506-99e4-6c3ff9ceea03 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:36.086620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce825b-119b-4a6b-be88-47199ccc54af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:36.088439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce825b-119b-4a6b-be88-47199ccc54af None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-6651b78f-9d42-4506-99e4-6c3ff9ceea03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:36.099624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:36.100496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:36.101379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:36.101379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:36.101379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:36.101379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:36.101379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 293/593] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:36.106895 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b78f-9d42-4506-99e4-6c3ff9ceea03 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-73ce825b-119b-4a6b-be88-47199ccc54af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:36.107028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:36.108017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:36.109210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:36.109210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:36.109210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:36.109210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:36.109210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:36.109210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:36.109210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:36.109210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:36.111558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 301/594] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:36.112654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce825b-119b-4a6b-be88-47199ccc54af None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:36.155896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:36.213750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:36.226654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b78f-9d42-4506-99e4-6c3ff9ceea03 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:36.233299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce825b-119b-4a6b-be88-47199ccc54af None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:36.263774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13869e21-a15e-46ab-b23a-5b832ad5a94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:36.265132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13869e21-a15e-46ab-b23a-5b832ad5a94e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.265615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13869e21-a15e-46ab-b23a-5b832ad5a94e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.266105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13869e21-a15e-46ab-b23a-5b832ad5a94e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.273818 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520108a-8fcf-40ff-b940-3125cb29b5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:36.274507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520108a-8fcf-40ff-b940-3125cb29b5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.274791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520108a-8fcf-40ff-b940-3125cb29b5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.275116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520108a-8fcf-40ff-b940-3125cb29b5f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.276265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520108a-8fcf-40ff-b940-3125cb29b5f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:36.276573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520108a-8fcf-40ff-b940-3125cb29b5f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:36.277259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13869e21-a15e-46ab-b23a-5b832ad5a94e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:36.278020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520108a-8fcf-40ff-b940-3125cb29b5f7 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:36.279207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13869e21-a15e-46ab-b23a-5b832ad5a94e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:36.296925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 302/595] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:36.296925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520108a-8fcf-40ff-b940-3125cb29b5f7 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-13869e21-a15e-46ab-b23a-5b832ad5a94e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:36.301018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:36.301922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:36.302893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:36.302893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:36.302893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:36.302893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:36.302893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:36.302893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:36.302893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:36.302893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:36.302893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 05:41:36.302893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:36.310338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 294/596] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:36.311805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13869e21-a15e-46ab-b23a-5b832ad5a94e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:36.414400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520108a-8fcf-40ff-b940-3125cb29b5f7 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:36.435312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:36.441368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13869e21-a15e-46ab-b23a-5b832ad5a94e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:36.452636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf0042-4755-4cdb-9c79-6183d59d1a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:36.453717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf0042-4755-4cdb-9c79-6183d59d1a7e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.454194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf0042-4755-4cdb-9c79-6183d59d1a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.454690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf0042-4755-4cdb-9c79-6183d59d1a7e None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.455766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf0042-4755-4cdb-9c79-6183d59d1a7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:36.456317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf0042-4755-4cdb-9c79-6183d59d1a7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:36.458130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf0042-4755-4cdb-9c79-6183d59d1a7e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:36.471492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 303/597] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:36.472947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf0042-4755-4cdb-9c79-6183d59d1a7e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:36.477872 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:36.482856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026b96d-8b7e-4317-94b3-a072dfa7a337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:36.483701 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b96d-8b7e-4317-94b3-a072dfa7a337 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.483986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b96d-8b7e-4317-94b3-a072dfa7a337 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.484247 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b96d-8b7e-4317-94b3-a072dfa7a337 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.492553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026b96d-8b7e-4317-94b3-a072dfa7a337 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:36.494573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026b96d-8b7e-4317-94b3-a072dfa7a337 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-f026b96d-8b7e-4317-94b3-a072dfa7a337 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:36.511081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:36.512096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:36.512977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:36.512977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:36.512977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:36.512977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:36.512977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:36.512977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:36.512977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:36.512977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:36.515325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 295/598] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:36.519293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026b96d-8b7e-4317-94b3-a072dfa7a337 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:36.608246 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf0042-4755-4cdb-9c79-6183d59d1a7e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:36.642337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026b96d-8b7e-4317-94b3-a072dfa7a337 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:36.646886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23317ce-b5f9-4401-b3a8-120793f86487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:36.647854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23317ce-b5f9-4401-b3a8-120793f86487 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.648328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23317ce-b5f9-4401-b3a8-120793f86487 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.648807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23317ce-b5f9-4401-b3a8-120793f86487 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.650450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23317ce-b5f9-4401-b3a8-120793f86487 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:36.650984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23317ce-b5f9-4401-b3a8-120793f86487 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:36.652670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23317ce-b5f9-4401-b3a8-120793f86487 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:36.678831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 304/599] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Jul 4 05:41:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:36.680160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23317ce-b5f9-4401-b3a8-120793f86487 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:36.687409 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6f0ff-356e-4c63-a2cf-87e87199fd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:36.688323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6f0ff-356e-4c63-a2cf-87e87199fd29 None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.688599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6f0ff-356e-4c63-a2cf-87e87199fd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.688869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6f0ff-356e-4c63-a2cf-87e87199fd29 None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/ab33473d40094ec681e538cde250ec2a/roles/04f8b72053764c7f8067d4b478d31e95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.705652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6f0ff-356e-4c63-a2cf-87e87199fd29 None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=ab33473d40094ec681e538cde250ec2a, role_id=04f8b72053764c7f8067d4b478d31e95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:36.707210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6f0ff-356e-4c63-a2cf-87e87199fd29 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:36.731412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 296/600] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:41:36 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/ab33473d40094ec681e538cde250ec2a/roles/04f8b72053764c7f8067d4b478d31e95 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:36.731412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6f0ff-356e-4c63-a2cf-87e87199fd29 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:36.756352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:36.784813 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:36.831648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23317ce-b5f9-4401-b3a8-120793f86487 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:36.870341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b143e-3fe0-4582-a779-829ff812e0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:36.871065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b143e-3fe0-4582-a779-829ff812e0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.871424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b143e-3fe0-4582-a779-829ff812e0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.871699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b143e-3fe0-4582-a779-829ff812e0c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.872505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b143e-3fe0-4582-a779-829ff812e0c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:36.872812 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b143e-3fe0-4582-a779-829ff812e0c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:36.874308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b143e-3fe0-4582-a779-829ff812e0c6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:36.886062 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6f0ff-356e-4c63-a2cf-87e87199fd29 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:36.888043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 305/601] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:36.889413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c87d1f-7a15-4273-b1d1-191cfd10e01c None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.889696 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c87d1f-7a15-4273-b1d1-191cfd10e01c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.889969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c87d1f-7a15-4273-b1d1-191cfd10e01c None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.895506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 306/602] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:36.896217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c87d1f-7a15-4273-b1d1-191cfd10e01c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:36.927591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc2ea8-51df-49bd-a10e-36a09aedb15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:36.929048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc2ea8-51df-49bd-a10e-36a09aedb15f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.929278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc2ea8-51df-49bd-a10e-36a09aedb15f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.929557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc2ea8-51df-49bd-a10e-36a09aedb15f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:36.943619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc2ea8-51df-49bd-a10e-36a09aedb15f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:36.946325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc2ea8-51df-49bd-a10e-36a09aedb15f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-77dc2ea8-51df-49bd-a10e-36a09aedb15f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:36.972093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:36.973030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:36.973887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:36.973887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:36.973887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:36.973887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:36.974609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 297/603] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:36.980197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789c212-dce3-4235-b1fa-e06e0f260bf1 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:36.981917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789c212-dce3-4235-b1fa-e06e0f260bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:36.981917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789c212-dce3-4235-b1fa-e06e0f260bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.005028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-0789c212-dce3-4235-b1fa-e06e0f260bf1 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:37.008078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0789c212-dce3-4235-b1fa-e06e0f260bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:37.014661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c87d1f-7a15-4273-b1d1-191cfd10e01c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:37.040281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:37.055158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5a69f-c2e2-4bca-b9a0-12f0b3132b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:37.056141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5a69f-c2e2-4bca-b9a0-12f0b3132b5f None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:37.056476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5a69f-c2e2-4bca-b9a0-12f0b3132b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:37.056841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5a69f-c2e2-4bca-b9a0-12f0b3132b5f None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/0f63e110260e4471b0a54576d3573583/roles/04f8b72053764c7f8067d4b478d31e95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.072501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5a69f-c2e2-4bca-b9a0-12f0b3132b5f None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=0f63e110260e4471b0a54576d3573583, role_id=04f8b72053764c7f8067d4b478d31e95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:37.074027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5a69f-c2e2-4bca-b9a0-12f0b3132b5f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:37.085776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:37.092626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 307/604] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:41:36 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/0f63e110260e4471b0a54576d3573583/roles/04f8b72053764c7f8067d4b478d31e95 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:37.094216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5a69f-c2e2-4bca-b9a0-12f0b3132b5f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:37.104256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 298/605] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:37.127080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789c212-dce3-4235-b1fa-e06e0f260bf1 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:37.281529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5a69f-c2e2-4bca-b9a0-12f0b3132b5f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:37.307075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789c212-dce3-4235-b1fa-e06e0f260bf1 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:37.326514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6bbaa-eca8-4b60-9ada-d04bfbe11b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:37.327199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6bbaa-eca8-4b60-9ada-d04bfbe11b1c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:37.327510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6bbaa-eca8-4b60-9ada-d04bfbe11b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:37.327780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6bbaa-eca8-4b60-9ada-d04bfbe11b1c None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.328513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6bbaa-eca8-4b60-9ada-d04bfbe11b1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:37.328816 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6bbaa-eca8-4b60-9ada-d04bfbe11b1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:37.330207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6bbaa-eca8-4b60-9ada-d04bfbe11b1c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:37.343085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 308/606] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:37.345735 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24211f1-4c8a-478c-980f-11a43f798f50 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:37.346194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24211f1-4c8a-478c-980f-11a43f798f50 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:37.346625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24211f1-4c8a-478c-980f-11a43f798f50 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.348896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 309/607] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:37.349748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24211f1-4c8a-478c-980f-11a43f798f50 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:37.359293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa03df-c00b-4a55-a76b-7f350292ea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:37.360785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa03df-c00b-4a55-a76b-7f350292ea36 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:37.361174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa03df-c00b-4a55-a76b-7f350292ea36 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:37.361550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa03df-c00b-4a55-a76b-7f350292ea36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.375505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa03df-c00b-4a55-a76b-7f350292ea36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:37.384478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-0caa03df-c00b-4a55-a76b-7f350292ea36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:37.386451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:37.387727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:37.388792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:37.390926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 299/608] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:37.392729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49761026-dfb0-4468-8699-e98ea7c8a9de None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:37.393126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49761026-dfb0-4468-8699-e98ea7c8a9de None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:37.393500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49761026-dfb0-4468-8699-e98ea7c8a9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.420823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-49761026-dfb0-4468-8699-e98ea7c8a9de None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:37.422757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-49761026-dfb0-4468-8699-e98ea7c8a9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:37.467196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24211f1-4c8a-478c-980f-11a43f798f50 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:37.499639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:37.509879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 300/609] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:37.515551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49761026-dfb0-4468-8699-e98ea7c8a9de None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:37.521068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92bafa-3df4-40e7-b017-e7f6498963d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:37.522123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92bafa-3df4-40e7-b017-e7f6498963d4 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:37.522552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92bafa-3df4-40e7-b017-e7f6498963d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:37.524663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92bafa-3df4-40e7-b017-e7f6498963d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.536606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92bafa-3df4-40e7-b017-e7f6498963d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:37.538171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92bafa-3df4-40e7-b017-e7f6498963d4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-7f92bafa-3df4-40e7-b017-e7f6498963d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:37.555563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:37.556850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:37.557718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:37.557718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:37.557718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:37.557718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:37.557718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:37.557718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:37.557718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:37.557718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:37.562053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 310/610] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:37.562948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92bafa-3df4-40e7-b017-e7f6498963d4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:37.628304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49761026-dfb0-4468-8699-e98ea7c8a9de None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:37.667634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ab0fb-c636-4dca-8017-b3f45c1f10ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:37.669142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ab0fb-c636-4dca-8017-b3f45c1f10ae None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:37.669142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ab0fb-c636-4dca-8017-b3f45c1f10ae None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:37.669142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ab0fb-c636-4dca-8017-b3f45c1f10ae None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.675268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ab0fb-c636-4dca-8017-b3f45c1f10ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:37.675268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ab0fb-c636-4dca-8017-b3f45c1f10ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:37.675268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ab0fb-c636-4dca-8017-b3f45c1f10ae None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:37.675748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92bafa-3df4-40e7-b017-e7f6498963d4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:37.676536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:37.680850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 301/611] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:37.682047 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ab0fb-c636-4dca-8017-b3f45c1f10ae None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:37.722410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319ec40-a8f0-45a4-bc8c-1a9029b02f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:37.723280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319ec40-a8f0-45a4-bc8c-1a9029b02f29 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:37.723571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319ec40-a8f0-45a4-bc8c-1a9029b02f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:37.723845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319ec40-a8f0-45a4-bc8c-1a9029b02f29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.736770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319ec40-a8f0-45a4-bc8c-1a9029b02f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:37.743621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-1319ec40-a8f0-45a4-bc8c-1a9029b02f29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:37.747896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:37.748828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:37.749676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:37.751322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 311/612] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:37.752348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319ec40-a8f0-45a4-bc8c-1a9029b02f29 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:37.797623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ab0fb-c636-4dca-8017-b3f45c1f10ae None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:37.834028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6247d-6b4c-4bbb-bf40-19d472567b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:37.834856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6247d-6b4c-4bbb-bf40-19d472567b2c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:37.835085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6247d-6b4c-4bbb-bf40-19d472567b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:37.835396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6247d-6b4c-4bbb-bf40-19d472567b2c None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.836240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6247d-6b4c-4bbb-bf40-19d472567b2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:37.836512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6247d-6b4c-4bbb-bf40-19d472567b2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:37.838030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6247d-6b4c-4bbb-bf40-19d472567b2c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:37.856449 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 302/613] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:37.857343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6247d-6b4c-4bbb-bf40-19d472567b2c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:37.858661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319ec40-a8f0-45a4-bc8c-1a9029b02f29 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:37.893663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405db660-c19d-4096-aefa-520816d53ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:37.894863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405db660-c19d-4096-aefa-520816d53ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:37.895380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405db660-c19d-4096-aefa-520816d53ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:37.895856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405db660-c19d-4096-aefa-520816d53ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:37.907488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405db660-c19d-4096-aefa-520816d53ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:37.909272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405db660-c19d-4096-aefa-520816d53ec6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-405db660-c19d-4096-aefa-520816d53ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:37.927358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:37.928309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:37.929179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:37.929179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:37.929179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:37.929179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:37.931701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 312/614] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:37.932899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405db660-c19d-4096-aefa-520816d53ec6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:37.937366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:37.969868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6247d-6b4c-4bbb-bf40-19d472567b2c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:37.999632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:38.008259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd3ba3-8e01-42b1-86d4-5a55b60461c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:38.009244 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3ba3-8e01-42b1-86d4-5a55b60461c7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:38.009759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3ba3-8e01-42b1-86d4-5a55b60461c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:38.010250 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3ba3-8e01-42b1-86d4-5a55b60461c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:38.012300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd3ba3-8e01-42b1-86d4-5a55b60461c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:38.012765 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd3ba3-8e01-42b1-86d4-5a55b60461c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:38.014429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd3ba3-8e01-42b1-86d4-5a55b60461c7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:38.028652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 303/615] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:38.029798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd3ba3-8e01-42b1-86d4-5a55b60461c7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:38.039328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405db660-c19d-4096-aefa-520816d53ec6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:38.074682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982e7ab-526d-4126-a60c-4b173081e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:38.075698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982e7ab-526d-4126-a60c-4b173081e0cb None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:38.076076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982e7ab-526d-4126-a60c-4b173081e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:38.076661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982e7ab-526d-4126-a60c-4b173081e0cb None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/817b1a026f4b443daea514338ad8531a/roles/04f8b72053764c7f8067d4b478d31e95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:38.091425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982e7ab-526d-4126-a60c-4b173081e0cb None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=817b1a026f4b443daea514338ad8531a, role_id=04f8b72053764c7f8067d4b478d31e95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:38.092987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982e7ab-526d-4126-a60c-4b173081e0cb None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:38.112894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 313/616] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:41:37 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/817b1a026f4b443daea514338ad8531a/roles/04f8b72053764c7f8067d4b478d31e95 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:38.120071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982e7ab-526d-4126-a60c-4b173081e0cb None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:38.151203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd3ba3-8e01-42b1-86d4-5a55b60461c7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:38.195512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0236161-98e4-42d0-9dac-55c4371cee45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:38.197135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0236161-98e4-42d0-9dac-55c4371cee45 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:38.197616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0236161-98e4-42d0-9dac-55c4371cee45 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:38.198150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0236161-98e4-42d0-9dac-55c4371cee45 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:38.199239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0236161-98e4-42d0-9dac-55c4371cee45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:38.199985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0236161-98e4-42d0-9dac-55c4371cee45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:38.201886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0236161-98e4-42d0-9dac-55c4371cee45 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:38.219365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 304/617] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:38.228467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0236161-98e4-42d0-9dac-55c4371cee45 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:38.259738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:38.278490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:38.363316 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982e7ab-526d-4126-a60c-4b173081e0cb None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:38.382242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0236161-98e4-42d0-9dac-55c4371cee45 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:38.405560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b61bf-ac0e-405a-bf6e-0bbf44365353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:38.407091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b61bf-ac0e-405a-bf6e-0bbf44365353 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:38.407552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b61bf-ac0e-405a-bf6e-0bbf44365353 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:38.408237 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b61bf-ac0e-405a-bf6e-0bbf44365353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:38.422168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b61bf-ac0e-405a-bf6e-0bbf44365353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:38.423862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795f2a6-cae0-481d-8114-496a9d104271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:38.424199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b61bf-ac0e-405a-bf6e-0bbf44365353 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:38.424690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795f2a6-cae0-481d-8114-496a9d104271 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:38.424974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795f2a6-cae0-481d-8114-496a9d104271 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:38.425250 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795f2a6-cae0-481d-8114-496a9d104271 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:38.440186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795f2a6-cae0-481d-8114-496a9d104271 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:38.441725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795f2a6-cae0-481d-8114-496a9d104271 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-9795f2a6-cae0-481d-8114-496a9d104271 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:38.464629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:38.465625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:38.466658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 305/618] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-112b61bf-ac0e-405a-bf6e-0bbf44365353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:38.467275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:38.468066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:38.469116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:38.469116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:38.469116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:38.469116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:38.469116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 314/619] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:38.475874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b61bf-ac0e-405a-bf6e-0bbf44365353 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:38.476085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795f2a6-cae0-481d-8114-496a9d104271 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:38.618316 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:38.648881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795f2a6-cae0-481d-8114-496a9d104271 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:38.689772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b61bf-ac0e-405a-bf6e-0bbf44365353 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:38.701829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:38.721189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee3649-3834-4997-bd10-0bbd7aa60885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:38.721872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee3649-3834-4997-bd10-0bbd7aa60885 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:38.722972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee3649-3834-4997-bd10-0bbd7aa60885 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:38.726005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee3649-3834-4997-bd10-0bbd7aa60885 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:38.726751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee3649-3834-4997-bd10-0bbd7aa60885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:38.727063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee3649-3834-4997-bd10-0bbd7aa60885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:38.729534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee3649-3834-4997-bd10-0bbd7aa60885 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:38.757669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 306/620] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:38.758401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51352a09-1036-4271-bb3c-50cd5e9bda29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:38.759032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51352a09-1036-4271-bb3c-50cd5e9bda29 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:38.759380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51352a09-1036-4271-bb3c-50cd5e9bda29 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:38.759695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51352a09-1036-4271-bb3c-50cd5e9bda29 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:38.760554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51352a09-1036-4271-bb3c-50cd5e9bda29 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:38.760859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51352a09-1036-4271-bb3c-50cd5e9bda29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:38.762336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51352a09-1036-4271-bb3c-50cd5e9bda29 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:38.765605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee3649-3834-4997-bd10-0bbd7aa60885 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:38.805380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 315/621] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 4 05:41:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:38.810889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51352a09-1036-4271-bb3c-50cd5e9bda29 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:38.911395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee3649-3834-4997-bd10-0bbd7aa60885 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:38.951958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006eab86-3d61-4065-b9c6-dbf8a53489c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:38.952781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eab86-3d61-4065-b9c6-dbf8a53489c5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:38.953063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eab86-3d61-4065-b9c6-dbf8a53489c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:38.953338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eab86-3d61-4065-b9c6-dbf8a53489c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:38.962784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006eab86-3d61-4065-b9c6-dbf8a53489c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:38.964294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006eab86-3d61-4065-b9c6-dbf8a53489c5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-006eab86-3d61-4065-b9c6-dbf8a53489c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:38.979155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:38.980147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:38.981103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 307/622] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 4 05:41:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:38.982189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50424a47-0f9e-42d7-89ce-c4680eceb800 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:38.982295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50424a47-0f9e-42d7-89ce-c4680eceb800 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:38.982560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50424a47-0f9e-42d7-89ce-c4680eceb800 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:38.988848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 308/623] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:38.989665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50424a47-0f9e-42d7-89ce-c4680eceb800 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:38.993685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:39.006221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51352a09-1036-4271-bb3c-50cd5e9bda29 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:39.043086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbd5d9-b9d6-4eac-a740-e76833d4f793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:39.044388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbd5d9-b9d6-4eac-a740-e76833d4f793 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.044850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbd5d9-b9d6-4eac-a740-e76833d4f793 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.045304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbd5d9-b9d6-4eac-a740-e76833d4f793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.056533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbd5d9-b9d6-4eac-a740-e76833d4f793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:39.058480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbd5d9-b9d6-4eac-a740-e76833d4f793 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-0fbbd5d9-b9d6-4eac-a740-e76833d4f793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:39.077902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:39.078849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:39.079780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:39.079780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:39.079780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:39.079780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:39.082199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 316/624] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:39.083761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55186-a348-4449-a7dd-7a83a8aa9e7b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.084350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55186-a348-4449-a7dd-7a83a8aa9e7b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.084823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55186-a348-4449-a7dd-7a83a8aa9e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.098126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50424a47-0f9e-42d7-89ce-c4680eceb800 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:39.102910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-73f55186-a348-4449-a7dd-7a83a8aa9e7b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:39.104455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-73f55186-a348-4449-a7dd-7a83a8aa9e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:39.123640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:39.133533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd66f38-9c7a-495f-83f5-97e34a34468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:39.134229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd66f38-9c7a-495f-83f5-97e34a34468e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.134606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd66f38-9c7a-495f-83f5-97e34a34468e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.135006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd66f38-9c7a-495f-83f5-97e34a34468e None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.135972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd66f38-9c7a-495f-83f5-97e34a34468e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:39.136337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd66f38-9c7a-495f-83f5-97e34a34468e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:39.137902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd66f38-9c7a-495f-83f5-97e34a34468e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:39.161320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 309/625] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 4 05:41:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:39.162277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd66f38-9c7a-495f-83f5-97e34a34468e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:39.183386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 317/626] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:39.184739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f55186-a348-4449-a7dd-7a83a8aa9e7b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:39.230570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:39.270064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd66f38-9c7a-495f-83f5-97e34a34468e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:39.305191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ce0d1-14db-4024-87c5-e09c74a1fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:39.305832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ce0d1-14db-4024-87c5-e09c74a1fa6a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.306347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ce0d1-14db-4024-87c5-e09c74a1fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.306629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ce0d1-14db-4024-87c5-e09c74a1fa6a None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.307392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ce0d1-14db-4024-87c5-e09c74a1fa6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:39.307701 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ce0d1-14db-4024-87c5-e09c74a1fa6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:39.308621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f55186-a348-4449-a7dd-7a83a8aa9e7b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:39.309083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ce0d1-14db-4024-87c5-e09c74a1fa6a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:39.321312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 310/627] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:39.322247 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ce0d1-14db-4024-87c5-e09c74a1fa6a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:39.345066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098a336-3ce5-4eca-9797-5d0d0ae72d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:39.345819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098a336-3ce5-4eca-9797-5d0d0ae72d7f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.346195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098a336-3ce5-4eca-9797-5d0d0ae72d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.346542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098a336-3ce5-4eca-9797-5d0d0ae72d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.364082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:39.417916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098a336-3ce5-4eca-9797-5d0d0ae72d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:39.419786 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098a336-3ce5-4eca-9797-5d0d0ae72d7f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:39.430194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ce0d1-14db-4024-87c5-e09c74a1fa6a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:39.457074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:39.469237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269520a-7ca5-48ec-8cfd-b33f321c78db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:39.470056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269520a-7ca5-48ec-8cfd-b33f321c78db None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.470332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269520a-7ca5-48ec-8cfd-b33f321c78db None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.470605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269520a-7ca5-48ec-8cfd-b33f321c78db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.481034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269520a-7ca5-48ec-8cfd-b33f321c78db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-0269520a-7ca5-48ec-8cfd-b33f321c78db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:39.488798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:39.489825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:39.490698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 311/628] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:39.491463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c097dc6-1212-4a2d-8993-a6aa7843d4ea None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.491860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c097dc6-1212-4a2d-8993-a6aa7843d4ea None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.491860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c097dc6-1212-4a2d-8993-a6aa7843d4ea None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.493220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 312/629] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:39.497296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c097dc6-1212-4a2d-8993-a6aa7843d4ea None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:39.538048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 318/630] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:39.538048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098a336-3ce5-4eca-9797-5d0d0ae72d7f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:39.584853 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:39.617147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c097dc6-1212-4a2d-8993-a6aa7843d4ea None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:39.640798 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098a336-3ce5-4eca-9797-5d0d0ae72d7f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:39.651055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498cc8b-4d77-4f5d-a68a-6a7a682d667e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:39.651977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498cc8b-4d77-4f5d-a68a-6a7a682d667e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.652265 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498cc8b-4d77-4f5d-a68a-6a7a682d667e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.652540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498cc8b-4d77-4f5d-a68a-6a7a682d667e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.659529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2498cc8b-4d77-4f5d-a68a-6a7a682d667e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:39.661109 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2498cc8b-4d77-4f5d-a68a-6a7a682d667e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:39.674468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d4060-c153-4ecc-a6f1-d7ff71ab2fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:39.675477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d4060-c153-4ecc-a6f1-d7ff71ab2fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.675897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d4060-c153-4ecc-a6f1-d7ff71ab2fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-2498cc8b-4d77-4f5d-a68a-6a7a682d667e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:39.676126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:39.676971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:39.684496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:39.684496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:39.684496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:39.684496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:39.684496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:39.684496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:39.684496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:39.684496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:39.684969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d4060-c153-4ecc-a6f1-d7ff71ab2fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.684969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d4060-c153-4ecc-a6f1-d7ff71ab2fe6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:39.685106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 313/631] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:39.685174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d4060-c153-4ecc-a6f1-d7ff71ab2fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:39.685240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293982a-67ef-4f42-ac74-32867555fb49 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.685240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293982a-67ef-4f42-ac74-32867555fb49 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.685376 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d4060-c153-4ecc-a6f1-d7ff71ab2fe6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:39.685444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293982a-67ef-4f42-ac74-32867555fb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.694112 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 319/632] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:39.694112 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d4060-c153-4ecc-a6f1-d7ff71ab2fe6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:39.703135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a293982a-67ef-4f42-ac74-32867555fb49 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:39.704781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a293982a-67ef-4f42-ac74-32867555fb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:39.742922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:39.776660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 314/633] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:39.777501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293982a-67ef-4f42-ac74-32867555fb49 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:39.818706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d4060-c153-4ecc-a6f1-d7ff71ab2fe6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:39.829065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:39.855737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16ed3d-ca9b-4fc8-b913-af29196e3bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:39.856534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16ed3d-ca9b-4fc8-b913-af29196e3bfe None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.856807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16ed3d-ca9b-4fc8-b913-af29196e3bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.857100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16ed3d-ca9b-4fc8-b913-af29196e3bfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.868219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16ed3d-ca9b-4fc8-b913-af29196e3bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:39.869846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16ed3d-ca9b-4fc8-b913-af29196e3bfe None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-ff16ed3d-ca9b-4fc8-b913-af29196e3bfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:39.887116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:39.888148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:39.889024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:39.889024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:39.889024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:39.889024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:39.889024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 320/634] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:39.889301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16ed3d-ca9b-4fc8-b913-af29196e3bfe None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:39.897037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293982a-67ef-4f42-ac74-32867555fb49 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:39.931591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd4df6-8a3c-47ab-9042-67a1c834e71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:39.932175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd4df6-8a3c-47ab-9042-67a1c834e71f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:39.932455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd4df6-8a3c-47ab-9042-67a1c834e71f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:39.932725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd4df6-8a3c-47ab-9042-67a1c834e71f None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:39.933520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd4df6-8a3c-47ab-9042-67a1c834e71f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:39.933823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd4df6-8a3c-47ab-9042-67a1c834e71f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:39.935323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd4df6-8a3c-47ab-9042-67a1c834e71f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:39.952312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 315/635] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:39.952312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd4df6-8a3c-47ab-9042-67a1c834e71f None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:39.980140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:40.004861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16ed3d-ca9b-4fc8-b913-af29196e3bfe None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.041094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06267b7-4f1b-42e1-ab79-8a3e01e3c176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.041906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06267b7-4f1b-42e1-ab79-8a3e01e3c176 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.042187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06267b7-4f1b-42e1-ab79-8a3e01e3c176 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.042459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06267b7-4f1b-42e1-ab79-8a3e01e3c176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.053393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd4df6-8a3c-47ab-9042-67a1c834e71f None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.053637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06267b7-4f1b-42e1-ab79-8a3e01e3c176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:40.055201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06267b7-4f1b-42e1-ab79-8a3e01e3c176 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-b06267b7-4f1b-42e1-ab79-8a3e01e3c176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:40.075040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:40.076193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:40.077096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:40.077096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:40.077096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:40.077096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:40.077096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 321/636] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:40.077454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06267b7-4f1b-42e1-ab79-8a3e01e3c176 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:40.094241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:40.095213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927df43-2a2d-4b69-bdc5-3be35cb641c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.096122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927df43-2a2d-4b69-bdc5-3be35cb641c1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.096427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927df43-2a2d-4b69-bdc5-3be35cb641c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.096706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927df43-2a2d-4b69-bdc5-3be35cb641c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.107868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927df43-2a2d-4b69-bdc5-3be35cb641c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-8927df43-2a2d-4b69-bdc5-3be35cb641c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:40.115692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:40.116636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:40.117475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 316/637] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:40.118249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927df43-2a2d-4b69-bdc5-3be35cb641c1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:40.202404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06267b7-4f1b-42e1-ab79-8a3e01e3c176 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.223677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:40.233089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927df43-2a2d-4b69-bdc5-3be35cb641c1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.246415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f441c-f30e-4bd1-8070-adf233e2f0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.247100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f441c-f30e-4bd1-8070-adf233e2f0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.247412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f441c-f30e-4bd1-8070-adf233e2f0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.247689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f441c-f30e-4bd1-8070-adf233e2f0c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.248502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f441c-f30e-4bd1-8070-adf233e2f0c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:40.248805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f441c-f30e-4bd1-8070-adf233e2f0c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:40.250211 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f441c-f30e-4bd1-8070-adf233e2f0c2 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:40.267055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 322/638] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:40.267055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f441c-f30e-4bd1-8070-adf233e2f0c2 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:40.273164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218af62b-c03b-421b-a92c-da53662052d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.274049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af62b-c03b-421b-a92c-da53662052d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.274354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af62b-c03b-421b-a92c-da53662052d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.274621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af62b-c03b-421b-a92c-da53662052d2 None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/655b3c5b98fa4d91b328d04d42fc263a/roles/04f8b72053764c7f8067d4b478d31e95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.291173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218af62b-c03b-421b-a92c-da53662052d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=655b3c5b98fa4d91b328d04d42fc263a, role_id=04f8b72053764c7f8067d4b478d31e95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:40.292674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218af62b-c03b-421b-a92c-da53662052d2 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:40.319571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 317/639] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:41:40 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/655b3c5b98fa4d91b328d04d42fc263a/roles/04f8b72053764c7f8067d4b478d31e95 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:40.319571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218af62b-c03b-421b-a92c-da53662052d2 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:40.345577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:40.381481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f441c-f30e-4bd1-8070-adf233e2f0c2 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.419593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bb4d3-238b-49e4-b329-d85bcb233191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.420289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bb4d3-238b-49e4-b329-d85bcb233191 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.420559 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bb4d3-238b-49e4-b329-d85bcb233191 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.420836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bb4d3-238b-49e4-b329-d85bcb233191 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.421572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bb4d3-238b-49e4-b329-d85bcb233191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:40.421883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bb4d3-238b-49e4-b329-d85bcb233191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:40.423335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bb4d3-238b-49e4-b329-d85bcb233191 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:40.436274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 323/640] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:40.437154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bb4d3-238b-49e4-b329-d85bcb233191 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:40.458564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218af62b-c03b-421b-a92c-da53662052d2 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.499406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefe426-de6f-4dc6-bdb1-90f29a44e38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.500454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefe426-de6f-4dc6-bdb1-90f29a44e38f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.500736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefe426-de6f-4dc6-bdb1-90f29a44e38f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.501010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefe426-de6f-4dc6-bdb1-90f29a44e38f None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.501765 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefe426-de6f-4dc6-bdb1-90f29a44e38f None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:40.502069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefe426-de6f-4dc6-bdb1-90f29a44e38f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:40.503532 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefe426-de6f-4dc6-bdb1-90f29a44e38f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:40.512517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 318/641] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:40.513654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d716988-d4ee-4acf-8968-92fd4b71ba1d None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.513941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d716988-d4ee-4acf-8968-92fd4b71ba1d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.514212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d716988-d4ee-4acf-8968-92fd4b71ba1d None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.516842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 319/642] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:40.516842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d716988-d4ee-4acf-8968-92fd4b71ba1d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:40.522489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:40.558176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bb4d3-238b-49e4-b329-d85bcb233191 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.589164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:40.594945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a21c1-faad-4662-a631-9a15a0f3d4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.595793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a21c1-faad-4662-a631-9a15a0f3d4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.596350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a21c1-faad-4662-a631-9a15a0f3d4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.596629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a21c1-faad-4662-a631-9a15a0f3d4c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.603635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a21c1-faad-4662-a631-9a15a0f3d4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:40.605096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a21c1-faad-4662-a631-9a15a0f3d4c7 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-667a21c1-faad-4662-a631-9a15a0f3d4c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:40.619346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:40.620421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:40.621343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:40.621343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:40.621343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:40.621343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:40.621343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:40.621343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:40.621343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:40.621343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:40.621343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 324/643] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:40.621845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a21c1-faad-4662-a631-9a15a0f3d4c7 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:40.631670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d716988-d4ee-4acf-8968-92fd4b71ba1d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.665822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef5fa9-d687-4a5c-9b3c-498ecda1905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.666634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef5fa9-d687-4a5c-9b3c-498ecda1905c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.666909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef5fa9-d687-4a5c-9b3c-498ecda1905c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.667177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef5fa9-d687-4a5c-9b3c-498ecda1905c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.675470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef5fa9-d687-4a5c-9b3c-498ecda1905c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:40.676932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef5fa9-d687-4a5c-9b3c-498ecda1905c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-d4ef5fa9-d687-4a5c-9b3c-498ecda1905c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:40.691334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:40.692472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:40.693744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:40.693744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:40.693744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:40.693744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:40.693744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:40.693744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:40.693744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:40.693744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:40.693744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 05:41:40.693744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:40.697413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 320/644] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:40.698755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4daa7d-d7d9-448f-bc7f-27cbddc7a842 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.699028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4daa7d-d7d9-448f-bc7f-27cbddc7a842 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.699321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4daa7d-d7d9-448f-bc7f-27cbddc7a842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.716484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ef4daa7d-d7d9-448f-bc7f-27cbddc7a842 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:40.717811 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ef4daa7d-d7d9-448f-bc7f-27cbddc7a842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:40.735519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a21c1-faad-4662-a631-9a15a0f3d4c7 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.770552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f87c93-1e49-4e21-a8e4-b060810f59e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.771406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f87c93-1e49-4e21-a8e4-b060810f59e4 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.771690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f87c93-1e49-4e21-a8e4-b060810f59e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.771964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f87c93-1e49-4e21-a8e4-b060810f59e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.783655 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f87c93-1e49-4e21-a8e4-b060810f59e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:40.785123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f87c93-1e49-4e21-a8e4-b060810f59e4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:40.786089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-24f87c93-1e49-4e21-a8e4-b060810f59e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:40.804449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:40.805359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:40.806261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:40.806261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:40.806261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:40.806261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:40.806544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 321/645] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:40.806544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4daa7d-d7d9-448f-bc7f-27cbddc7a842 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:40.809036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 325/646] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:40.810086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f87c93-1e49-4e21-a8e4-b060810f59e4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:40.858946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:40.916182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f87c93-1e49-4e21-a8e4-b060810f59e4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.916412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4daa7d-d7d9-448f-bc7f-27cbddc7a842 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:40.952583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8388de-55a7-42fd-9e39-3b5ac88885a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.953664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8388de-55a7-42fd-9e39-3b5ac88885a0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.954118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8388de-55a7-42fd-9e39-3b5ac88885a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.954581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8388de-55a7-42fd-9e39-3b5ac88885a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.955571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8388de-55a7-42fd-9e39-3b5ac88885a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:40.956045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8388de-55a7-42fd-9e39-3b5ac88885a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:40.956301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154534b-b732-42ec-a3d2-c7c10a4533a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:40.956975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154534b-b732-42ec-a3d2-c7c10a4533a8 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:40.957249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154534b-b732-42ec-a3d2-c7c10a4533a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:40.957520 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154534b-b732-42ec-a3d2-c7c10a4533a8 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:40.958364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154534b-b732-42ec-a3d2-c7c10a4533a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:40.958561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8388de-55a7-42fd-9e39-3b5ac88885a0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:40.958656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154534b-b732-42ec-a3d2-c7c10a4533a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:40.960118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154534b-b732-42ec-a3d2-c7c10a4533a8 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:40.972717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 322/647] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:40.973761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8388de-55a7-42fd-9e39-3b5ac88885a0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:40.981027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 326/648] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 4 05:41:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:40.981785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154534b-b732-42ec-a3d2-c7c10a4533a8 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:41.034953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:41.081463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8388de-55a7-42fd-9e39-3b5ac88885a0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:41.091356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154534b-b732-42ec-a3d2-c7c10a4533a8 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:41.093011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:41.114540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36384973-d450-4f92-a253-b816d8939259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:41.115206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36384973-d450-4f92-a253-b816d8939259 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:41.115511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36384973-d450-4f92-a253-b816d8939259 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:41.115773 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36384973-d450-4f92-a253-b816d8939259 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:41.116490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36384973-d450-4f92-a253-b816d8939259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:41.116791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36384973-d450-4f92-a253-b816d8939259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:41.118154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36384973-d450-4f92-a253-b816d8939259 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:41.129833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 323/649] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:41.130468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36384973-d450-4f92-a253-b816d8939259 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:41.135532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dad5ce-4db8-4a75-a4cd-34725800aa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:41.136333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dad5ce-4db8-4a75-a4cd-34725800aa80 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:41.136605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dad5ce-4db8-4a75-a4cd-34725800aa80 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:41.136866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dad5ce-4db8-4a75-a4cd-34725800aa80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:41.147365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dad5ce-4db8-4a75-a4cd-34725800aa80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-25dad5ce-4db8-4a75-a4cd-34725800aa80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:41.154996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:41.155995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:41.156842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 327/650] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:41.157546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dad5ce-4db8-4a75-a4cd-34725800aa80 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:41.239726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36384973-d450-4f92-a253-b816d8939259 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:41.258693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:41.264845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dad5ce-4db8-4a75-a4cd-34725800aa80 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:41.282816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1b2aa-fab5-4425-9911-e8f4a742e6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:41.283510 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1b2aa-fab5-4425-9911-e8f4a742e6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:41.283791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1b2aa-fab5-4425-9911-e8f4a742e6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:41.284064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1b2aa-fab5-4425-9911-e8f4a742e6a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:41.284904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1b2aa-fab5-4425-9911-e8f4a742e6a3 None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=2cb245e4300e4d5fbdd425e936f4ac77) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:41.285207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1b2aa-fab5-4425-9911-e8f4a742e6a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:41.286642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1b2aa-fab5-4425-9911-e8f4a742e6a3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:41.303288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 324/651] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:41 2022] GET /identity/v3/role_assignments?user.id=2cb245e4300e4d5fbdd425e936f4ac77&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:41.303288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1b2aa-fab5-4425-9911-e8f4a742e6a3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:41.311528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8ddc6-ba5e-4730-91b0-07a74db96575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:41.312300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8ddc6-ba5e-4730-91b0-07a74db96575 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:41.312647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8ddc6-ba5e-4730-91b0-07a74db96575 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:41.312984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8ddc6-ba5e-4730-91b0-07a74db96575 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:41.347586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:41.395590 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8ddc6-ba5e-4730-91b0-07a74db96575 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:41.397270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8ddc6-ba5e-4730-91b0-07a74db96575 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:41.431536 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1b2aa-fab5-4425-9911-e8f4a742e6a3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:41.484012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f247d-c823-4547-b792-4d51b3801bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:41.484849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f247d-c823-4547-b792-4d51b3801bae None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:41.485587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f247d-c823-4547-b792-4d51b3801bae None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:41.485874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f247d-c823-4547-b792-4d51b3801bae None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:41.497055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f247d-c823-4547-b792-4d51b3801bae None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:41.498509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f247d-c823-4547-b792-4d51b3801bae None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-1b8f247d-c823-4547-b792-4d51b3801bae None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:41.513706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:41.514759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:41.515833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 325/652] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:41.517016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f247d-c823-4547-b792-4d51b3801bae None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:41.525777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 328/653] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:41.527457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8ddc6-ba5e-4730-91b0-07a74db96575 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:41.564396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:41.604918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:41.633119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f247d-c823-4547-b792-4d51b3801bae None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:41.657977 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8ddc6-ba5e-4730-91b0-07a74db96575 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:41.674922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796a432-12d4-4664-ba69-464360343691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:41.676071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796a432-12d4-4664-ba69-464360343691 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:41.676531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796a432-12d4-4664-ba69-464360343691 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:41.677280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796a432-12d4-4664-ba69-464360343691 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:41.678232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796a432-12d4-4664-ba69-464360343691 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:41.678720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796a432-12d4-4664-ba69-464360343691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:41.680732 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796a432-12d4-4664-ba69-464360343691 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:41.692448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 326/654] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:41.693520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796a432-12d4-4664-ba69-464360343691 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:41.701430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fe614-1063-4301-9873-f21680304ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:41.702192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fe614-1063-4301-9873-f21680304ced None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:41.702537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fe614-1063-4301-9873-f21680304ced None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:41.702894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fe614-1063-4301-9873-f21680304ced None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:41.703841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fe614-1063-4301-9873-f21680304ced None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:41.704212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fe614-1063-4301-9873-f21680304ced None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:41.706228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fe614-1063-4301-9873-f21680304ced None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:41.729984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 329/655] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 4 05:41:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:41.731155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fe614-1063-4301-9873-f21680304ced None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:41.806465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:41.810286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796a432-12d4-4664-ba69-464360343691 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:41.841191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fe614-1063-4301-9873-f21680304ced None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:41.845883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a4c0a-24ff-45bb-8fcb-c9db07582741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:41.846697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a4c0a-24ff-45bb-8fcb-c9db07582741 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:41.846974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a4c0a-24ff-45bb-8fcb-c9db07582741 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:41.847239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a4c0a-24ff-45bb-8fcb-c9db07582741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:41.858264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a4c0a-24ff-45bb-8fcb-c9db07582741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:41.859748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a4c0a-24ff-45bb-8fcb-c9db07582741 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:41.867221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-728a4c0a-24ff-45bb-8fcb-c9db07582741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:41.877738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:41.878663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:41.879591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:41.879591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:41.879591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:41.879591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:41.883748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 327/656] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:41.884524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a4c0a-24ff-45bb-8fcb-c9db07582741 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:41.886919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523a859-64df-40ce-b1f8-7c8e9cec4180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:41.888000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523a859-64df-40ce-b1f8-7c8e9cec4180 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:41.888490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523a859-64df-40ce-b1f8-7c8e9cec4180 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:41.888965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523a859-64df-40ce-b1f8-7c8e9cec4180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:41.900118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523a859-64df-40ce-b1f8-7c8e9cec4180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:41.901922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523a859-64df-40ce-b1f8-7c8e9cec4180 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-3523a859-64df-40ce-b1f8-7c8e9cec4180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:41.923794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:41.924750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:41.925632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:41.925632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:41.925632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:41.925632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:41.925632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 330/657] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:41.926371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523a859-64df-40ce-b1f8-7c8e9cec4180 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:41.995768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a4c0a-24ff-45bb-8fcb-c9db07582741 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:42.037878 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dc2e1-ff96-45a9-b4f4-fed0c01bfcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:42.038533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dc2e1-ff96-45a9-b4f4-fed0c01bfcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.038813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dc2e1-ff96-45a9-b4f4-fed0c01bfcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.039081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dc2e1-ff96-45a9-b4f4-fed0c01bfcd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.043829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523a859-64df-40ce-b1f8-7c8e9cec4180 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:42.062528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:42.081482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31502b51-4340-43f4-849b-470dc93c8d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:42.082260 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31502b51-4340-43f4-849b-470dc93c8d26 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.082600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31502b51-4340-43f4-849b-470dc93c8d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.082941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31502b51-4340-43f4-849b-470dc93c8d26 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.083865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31502b51-4340-43f4-849b-470dc93c8d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:42.084229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31502b51-4340-43f4-849b-470dc93c8d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:42.085800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31502b51-4340-43f4-849b-470dc93c8d26 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:42.099425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 331/658] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:42.100925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31502b51-4340-43f4-849b-470dc93c8d26 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:42.115274 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572dc2e1-ff96-45a9-b4f4-fed0c01bfcd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:42.116824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572dc2e1-ff96-45a9-b4f4-fed0c01bfcd8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:42.134104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:42.210722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31502b51-4340-43f4-849b-470dc93c8d26 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:42.224784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 328/659] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:42.225807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dc2e1-ff96-45a9-b4f4-fed0c01bfcd8 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:42.247994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8daf35-6022-461f-804d-d04e75868feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:42.249047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8daf35-6022-461f-804d-d04e75868feb None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.249530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8daf35-6022-461f-804d-d04e75868feb None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.250026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8daf35-6022-461f-804d-d04e75868feb None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.251171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8daf35-6022-461f-804d-d04e75868feb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:42.251698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8daf35-6022-461f-804d-d04e75868feb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:42.253300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8daf35-6022-461f-804d-d04e75868feb None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:42.266737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 332/660] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:42.267907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8daf35-6022-461f-804d-d04e75868feb None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:42.285783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:42.335679 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dc2e1-ff96-45a9-b4f4-fed0c01bfcd8 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:42.365350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:42.373215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ced6e2-e9ef-4a76-b87a-def6946454f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:42.374026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ced6e2-e9ef-4a76-b87a-def6946454f4 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.374300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ced6e2-e9ef-4a76-b87a-def6946454f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.374567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ced6e2-e9ef-4a76-b87a-def6946454f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.376578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8daf35-6022-461f-804d-d04e75868feb None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:42.381739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ced6e2-e9ef-4a76-b87a-def6946454f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:42.383203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ced6e2-e9ef-4a76-b87a-def6946454f4 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-a6ced6e2-e9ef-4a76-b87a-def6946454f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:42.397321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:42.398240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:42.399138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:42.399138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:42.399138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:42.399138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:42.399138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:42.399138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:42.399138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:42.399138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:42.399138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 329/661] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:42.399679 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ced6e2-e9ef-4a76-b87a-def6946454f4 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:42.416374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd8acb-3099-4f91-aab3-71c9e4ab8d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:42.417579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd8acb-3099-4f91-aab3-71c9e4ab8d05 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.418073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd8acb-3099-4f91-aab3-71c9e4ab8d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.418560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd8acb-3099-4f91-aab3-71c9e4ab8d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.429692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd8acb-3099-4f91-aab3-71c9e4ab8d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:42.431552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd8acb-3099-4f91-aab3-71c9e4ab8d05 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-b7dd8acb-3099-4f91-aab3-71c9e4ab8d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:42.451673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:42.452578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:42.453548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:42.453548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:42.453548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:42.453548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:42.455557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 333/662] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:42.457019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89612b4-ada3-4a87-a07f-372acbb95382 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.457528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89612b4-ada3-4a87-a07f-372acbb95382 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.458114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89612b4-ada3-4a87-a07f-372acbb95382 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.459894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 334/663] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:42.460778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89612b4-ada3-4a87-a07f-372acbb95382 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:42.510933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ced6e2-e9ef-4a76-b87a-def6946454f4 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:42.527700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:42.551378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87046e46-1155-4dce-819e-7cdd0e0d96ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:42.552204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046e46-1155-4dce-819e-7cdd0e0d96ec None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.552504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046e46-1155-4dce-819e-7cdd0e0d96ec None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.552778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046e46-1155-4dce-819e-7cdd0e0d96ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.561025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87046e46-1155-4dce-819e-7cdd0e0d96ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:42.562479 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87046e46-1155-4dce-819e-7cdd0e0d96ec None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:42.575912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89612b4-ada3-4a87-a07f-372acbb95382 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-87046e46-1155-4dce-819e-7cdd0e0d96ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:42.577275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:42.578164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:42.579080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 330/664] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:42.580156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87046e46-1155-4dce-819e-7cdd0e0d96ec None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:42.618362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f19608-f55c-46d7-a31d-3df2d8b30df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:42.620217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f19608-f55c-46d7-a31d-3df2d8b30df3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.620498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f19608-f55c-46d7-a31d-3df2d8b30df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.620768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f19608-f55c-46d7-a31d-3df2d8b30df3 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.621562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f19608-f55c-46d7-a31d-3df2d8b30df3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:42.621865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f19608-f55c-46d7-a31d-3df2d8b30df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:42.623333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f19608-f55c-46d7-a31d-3df2d8b30df3 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:42.635553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 335/665] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:42.636821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf6cc-c576-4610-9e34-dca894e326de None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.637108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf6cc-c576-4610-9e34-dca894e326de None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.637379 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf6cc-c576-4610-9e34-dca894e326de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.643054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:42.654622 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-96adf6cc-c576-4610-9e34-dca894e326de None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:42.655836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-96adf6cc-c576-4610-9e34-dca894e326de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:42.690714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87046e46-1155-4dce-819e-7cdd0e0d96ec None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:42.725067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5c9e1-de1d-494d-8e5c-1e2b686b93fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:42.725741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5c9e1-de1d-494d-8e5c-1e2b686b93fd None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.726021 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5c9e1-de1d-494d-8e5c-1e2b686b93fd None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.726524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5c9e1-de1d-494d-8e5c-1e2b686b93fd None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.727007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5c9e1-de1d-494d-8e5c-1e2b686b93fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:42.727336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5c9e1-de1d-494d-8e5c-1e2b686b93fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:42.728848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5c9e1-de1d-494d-8e5c-1e2b686b93fd None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:42.737190 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 336/666] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:42.738328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adf6cc-c576-4610-9e34-dca894e326de None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:42.751755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 331/667] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:42.753671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5c9e1-de1d-494d-8e5c-1e2b686b93fd None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:42.785842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:42.851654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adf6cc-c576-4610-9e34-dca894e326de None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:42.856288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5c9e1-de1d-494d-8e5c-1e2b686b93fd None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:42.887172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d973c46-1982-45f9-b45d-f04bf046456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:42.887815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d973c46-1982-45f9-b45d-f04bf046456b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.888078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d973c46-1982-45f9-b45d-f04bf046456b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.888345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d973c46-1982-45f9-b45d-f04bf046456b None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.889205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d973c46-1982-45f9-b45d-f04bf046456b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:42.889528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d973c46-1982-45f9-b45d-f04bf046456b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:42.890600 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63a335-04c1-44ca-9261-99796f9a1354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:42.890922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d973c46-1982-45f9-b45d-f04bf046456b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:42.891318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63a335-04c1-44ca-9261-99796f9a1354 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:42.891605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63a335-04c1-44ca-9261-99796f9a1354 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:42.891881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63a335-04c1-44ca-9261-99796f9a1354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:42.892685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:42.892787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63a335-04c1-44ca-9261-99796f9a1354 None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=ab33473d40094ec681e538cde250ec2a) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:42.893013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63a335-04c1-44ca-9261-99796f9a1354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:42.894724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63a335-04c1-44ca-9261-99796f9a1354 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:42.900870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 332/668] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:42 2022] GET /identity/v3/role_assignments?user.id=ab33473d40094ec681e538cde250ec2a&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:42.901660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63a335-04c1-44ca-9261-99796f9a1354 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:42.925499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 337/669] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Jul 4 05:41:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:42.927100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d973c46-1982-45f9-b45d-f04bf046456b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:43.025103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:43.034084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63a335-04c1-44ca-9261-99796f9a1354 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:43.052412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d973c46-1982-45f9-b45d-f04bf046456b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:43.085156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbea64-4aa3-475b-8ca7-77762eda56f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:43.086322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbea64-4aa3-475b-8ca7-77762eda56f8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.086600 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbea64-4aa3-475b-8ca7-77762eda56f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.086872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbea64-4aa3-475b-8ca7-77762eda56f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.093090 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdc2e0-9076-4c1d-8378-a21fdd7fc9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:43.094173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc2e0-9076-4c1d-8378-a21fdd7fc9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.094349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dbea64-4aa3-475b-8ca7-77762eda56f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:43.094480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc2e0-9076-4c1d-8378-a21fdd7fc9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.094711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc2e0-9076-4c1d-8378-a21fdd7fc9e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.096122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dbea64-4aa3-475b-8ca7-77762eda56f8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:43.106677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdc2e0-9076-4c1d-8378-a21fdd7fc9e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-95dbea64-4aa3-475b-8ca7-77762eda56f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:43.114249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:43.115122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:43.116061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:43.116061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:43.116061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:43.116061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:43.116061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:43.116061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:43.116061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:43.116061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-47fdc2e0-9076-4c1d-8378-a21fdd7fc9e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:43.116481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:43.117904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 338/671] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:43.118837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdc2e0-9076-4c1d-8378-a21fdd7fc9e2 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:43.120867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 333/671] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:43.121924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbea64-4aa3-475b-8ca7-77762eda56f8 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:43.186136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:43.245344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdc2e0-9076-4c1d-8378-a21fdd7fc9e2 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:43.255035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbea64-4aa3-475b-8ca7-77762eda56f8 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:43.289221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d3113-7158-4e93-bd63-f45d2dca9133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:43.289887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d3113-7158-4e93-bd63-f45d2dca9133 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.290106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aefcb0-6a21-4fb6-9071-5eae83fb86d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:43.290327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d3113-7158-4e93-bd63-f45d2dca9133 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.290420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d3113-7158-4e93-bd63-f45d2dca9133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.291385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aefcb0-6a21-4fb6-9071-5eae83fb86d0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.291855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aefcb0-6a21-4fb6-9071-5eae83fb86d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.292325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aefcb0-6a21-4fb6-9071-5eae83fb86d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.293319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aefcb0-6a21-4fb6-9071-5eae83fb86d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:43.293811 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aefcb0-6a21-4fb6-9071-5eae83fb86d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:43.295477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aefcb0-6a21-4fb6-9071-5eae83fb86d0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:43.308476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 334/672] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:43.309672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aefcb0-6a21-4fb6-9071-5eae83fb86d0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:43.328511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:43.379516 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d3113-7158-4e93-bd63-f45d2dca9133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:43.381288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d3113-7158-4e93-bd63-f45d2dca9133 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:43.420327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aefcb0-6a21-4fb6-9071-5eae83fb86d0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:43.443393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:43.469607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26e024-4eab-4ac8-b23b-80fb84e4d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:43.470373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26e024-4eab-4ac8-b23b-80fb84e4d417 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.470654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26e024-4eab-4ac8-b23b-80fb84e4d417 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.470927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26e024-4eab-4ac8-b23b-80fb84e4d417 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.471764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26e024-4eab-4ac8-b23b-80fb84e4d417 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:43.472070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26e024-4eab-4ac8-b23b-80fb84e4d417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:43.473905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26e024-4eab-4ac8-b23b-80fb84e4d417 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:43.490242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 335/673] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:43.491186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26e024-4eab-4ac8-b23b-80fb84e4d417 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:43.510984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 339/674] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:43.512385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d3113-7158-4e93-bd63-f45d2dca9133 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:43.582183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:43.607369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26e024-4eab-4ac8-b23b-80fb84e4d417 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:43.616918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d3113-7158-4e93-bd63-f45d2dca9133 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:43.642838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddd7fc-eb11-41dd-a02e-87d2035d0150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:43.643529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddd7fc-eb11-41dd-a02e-87d2035d0150 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.643823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddd7fc-eb11-41dd-a02e-87d2035d0150 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.644086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddd7fc-eb11-41dd-a02e-87d2035d0150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.644954 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddd7fc-eb11-41dd-a02e-87d2035d0150 None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=0f63e110260e4471b0a54576d3573583) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:43.645229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddd7fc-eb11-41dd-a02e-87d2035d0150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:43.646652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddd7fc-eb11-41dd-a02e-87d2035d0150 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:43.652805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 336/675] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:43 2022] GET /identity/v3/role_assignments?user.id=0f63e110260e4471b0a54576d3573583&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:43.653031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7c109-b9e5-4d54-82c2-8ae35aabfd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:43.653593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddd7fc-eb11-41dd-a02e-87d2035d0150 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:43.653796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c109-b9e5-4d54-82c2-8ae35aabfd82 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.654108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c109-b9e5-4d54-82c2-8ae35aabfd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.654341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c109-b9e5-4d54-82c2-8ae35aabfd82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.671300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7c109-b9e5-4d54-82c2-8ae35aabfd82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:43.672717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7c109-b9e5-4d54-82c2-8ae35aabfd82 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-c8b7c109-b9e5-4d54-82c2-8ae35aabfd82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:43.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:43.702024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:43.702939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:43.702939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:43.702939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:43.702939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:43.702939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 340/676] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:43.707176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848c7af-f2a2-47af-8a46-a8da56ff0546 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.707176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848c7af-f2a2-47af-8a46-a8da56ff0546 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.707176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848c7af-f2a2-47af-8a46-a8da56ff0546 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.707176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 341/677] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:43.708389 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848c7af-f2a2-47af-8a46-a8da56ff0546 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:43.712773 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:43.794066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddd7fc-eb11-41dd-a02e-87d2035d0150 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:43.826177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848c7af-f2a2-47af-8a46-a8da56ff0546 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:43.842213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c4fac-440f-4569-b9e1-ac7bac389049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:43.843218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c4fac-440f-4569-b9e1-ac7bac389049 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.843887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c4fac-440f-4569-b9e1-ac7bac389049 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.844172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c4fac-440f-4569-b9e1-ac7bac389049 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.856515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c4fac-440f-4569-b9e1-ac7bac389049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:43.857990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c4fac-440f-4569-b9e1-ac7bac389049 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:43.860026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:43.880269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441db92a-bf7d-42d8-910d-5a2f862fe153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:43.880269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db92a-bf7d-42d8-910d-5a2f862fe153 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-cc0c4fac-440f-4569-b9e1-ac7bac389049 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:43.880561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:43.881379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:43.882275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:43.882275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:43.882275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:43.882275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:43.882496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db92a-bf7d-42d8-910d-5a2f862fe153 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.882496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db92a-bf7d-42d8-910d-5a2f862fe153 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.882496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441db92a-bf7d-42d8-910d-5a2f862fe153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:43.882496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441db92a-bf7d-42d8-910d-5a2f862fe153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:43.882737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 337/678] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:43.882737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d8ddd-13fb-465c-948e-d5f8afac4dc1 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:43.882737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d8ddd-13fb-465c-948e-d5f8afac4dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:43.882737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d8ddd-13fb-465c-948e-d5f8afac4dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:43.882974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441db92a-bf7d-42d8-910d-5a2f862fe153 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:43.897761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 342/679] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:43.898613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441db92a-bf7d-42d8-910d-5a2f862fe153 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:43.905114 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-769d8ddd-13fb-465c-948e-d5f8afac4dc1 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:43.906336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-769d8ddd-13fb-465c-948e-d5f8afac4dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:43.973736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:43.997617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 338/680] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:43.998771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d8ddd-13fb-465c-948e-d5f8afac4dc1 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.015430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441db92a-bf7d-42d8-910d-5a2f862fe153 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:44.051073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd370201-ad06-4371-a166-f0b29b9a3c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:44.051957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd370201-ad06-4371-a166-f0b29b9a3c67 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:44.052373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd370201-ad06-4371-a166-f0b29b9a3c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:44.052745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd370201-ad06-4371-a166-f0b29b9a3c67 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:44.053625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd370201-ad06-4371-a166-f0b29b9a3c67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:44.054026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd370201-ad06-4371-a166-f0b29b9a3c67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:44.055532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd370201-ad06-4371-a166-f0b29b9a3c67 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:44.069061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 343/681] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:44.070457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd370201-ad06-4371-a166-f0b29b9a3c67 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.119486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d8ddd-13fb-465c-948e-d5f8afac4dc1 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:44.145273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:44.159408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc64b4-588b-4da3-90e9-fb06604a1474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:44.160589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc64b4-588b-4da3-90e9-fb06604a1474 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:44.161062 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc64b4-588b-4da3-90e9-fb06604a1474 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:44.161529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc64b4-588b-4da3-90e9-fb06604a1474 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:44.170663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc64b4-588b-4da3-90e9-fb06604a1474 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:44.172746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc64b4-588b-4da3-90e9-fb06604a1474 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-c7dc64b4-588b-4da3-90e9-fb06604a1474 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:44.189781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:44.190775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:44.191783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:44.191783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:44.191783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:44.191783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:44.191783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:44.191783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:44.191783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:44.191783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:44.191783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:41:44.191783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:44.192339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd370201-ad06-4371-a166-f0b29b9a3c67 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:44.195475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 339/682] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:44.196549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc64b4-588b-4da3-90e9-fb06604a1474 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.222967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:44.240653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87144e-2d13-42f6-9236-d33f40692630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:44.241525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87144e-2d13-42f6-9236-d33f40692630 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:44.241806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87144e-2d13-42f6-9236-d33f40692630 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:44.242079 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87144e-2d13-42f6-9236-d33f40692630 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:44.255484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87144e-2d13-42f6-9236-d33f40692630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-0d87144e-2d13-42f6-9236-d33f40692630 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:44.264236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:44.265176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:44.266619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 344/683] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:44.267357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87144e-2d13-42f6-9236-d33f40692630 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.318641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc64b4-588b-4da3-90e9-fb06604a1474 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:44.363126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168631df-0dbb-4a31-9cc6-a64260878832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:44.364284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168631df-0dbb-4a31-9cc6-a64260878832 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:44.364845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168631df-0dbb-4a31-9cc6-a64260878832 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:44.365321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168631df-0dbb-4a31-9cc6-a64260878832 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:44.372838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168631df-0dbb-4a31-9cc6-a64260878832 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:44.374554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168631df-0dbb-4a31-9cc6-a64260878832 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:44.384615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87144e-2d13-42f6-9236-d33f40692630 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-168631df-0dbb-4a31-9cc6-a64260878832 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:44.389778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:44.390642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:44.391579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:44.391579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:44.391579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:44.391579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:44.391579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:44.391579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:44.391579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:44.391579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:44.394189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 340/684] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:44.395387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168631df-0dbb-4a31-9cc6-a64260878832 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.420814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6e82b-64f1-4a79-9047-cd72c9e49a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:44.421524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6e82b-64f1-4a79-9047-cd72c9e49a1f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:44.421802 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6e82b-64f1-4a79-9047-cd72c9e49a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:44.422072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6e82b-64f1-4a79-9047-cd72c9e49a1f None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:44.423032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6e82b-64f1-4a79-9047-cd72c9e49a1f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:44.423365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6e82b-64f1-4a79-9047-cd72c9e49a1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:44.424981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6e82b-64f1-4a79-9047-cd72c9e49a1f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:44.447652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:44.447878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 345/685] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 4 05:41:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:44.448980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6e82b-64f1-4a79-9047-cd72c9e49a1f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.499596 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:44.505658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168631df-0dbb-4a31-9cc6-a64260878832 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:44.545829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337430f7-938a-4e7b-bd7a-e9feae96d0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:44.546889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337430f7-938a-4e7b-bd7a-e9feae96d0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:44.547490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337430f7-938a-4e7b-bd7a-e9feae96d0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:44.547956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337430f7-938a-4e7b-bd7a-e9feae96d0e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:44.548895 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337430f7-938a-4e7b-bd7a-e9feae96d0e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:44.549948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337430f7-938a-4e7b-bd7a-e9feae96d0e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:44.551703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337430f7-938a-4e7b-bd7a-e9feae96d0e5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:44.559981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6e82b-64f1-4a79-9047-cd72c9e49a1f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:44.565123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 341/686] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:44.566226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337430f7-938a-4e7b-bd7a-e9feae96d0e5 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.597899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd04c9f-5cad-4de7-83db-424e0d2878d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:44.598708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04c9f-5cad-4de7-83db-424e0d2878d3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:44.598993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04c9f-5cad-4de7-83db-424e0d2878d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:44.599274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04c9f-5cad-4de7-83db-424e0d2878d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:44.600193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd04c9f-5cad-4de7-83db-424e0d2878d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:44.600481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd04c9f-5cad-4de7-83db-424e0d2878d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:44.601942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd04c9f-5cad-4de7-83db-424e0d2878d3 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:44.614995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 346/687] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:44.616043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd04c9f-5cad-4de7-83db-424e0d2878d3 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.672382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337430f7-938a-4e7b-bd7a-e9feae96d0e5 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:44.680228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:44.710011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952ca20-7e61-4e47-98d7-fadcc16dc6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:44.711095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952ca20-7e61-4e47-98d7-fadcc16dc6eb None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:44.711658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952ca20-7e61-4e47-98d7-fadcc16dc6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:44.712179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952ca20-7e61-4e47-98d7-fadcc16dc6eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:44.724193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd04c9f-5cad-4de7-83db-424e0d2878d3 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:44.725981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:44.761665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2a912-f524-429a-9977-3dc93a67b54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:44.762387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2a912-f524-429a-9977-3dc93a67b54a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:44.762666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2a912-f524-429a-9977-3dc93a67b54a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:44.762939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2a912-f524-429a-9977-3dc93a67b54a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:44.774807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2a912-f524-429a-9977-3dc93a67b54a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:44.776338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2a912-f524-429a-9977-3dc93a67b54a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:44.787466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952ca20-7e61-4e47-98d7-fadcc16dc6eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:44.789196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952ca20-7e61-4e47-98d7-fadcc16dc6eb None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-46c2a912-f524-429a-9977-3dc93a67b54a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:44.794720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:44.795631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:44.797075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:44.797075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:44.797075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:44.797075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:44.797075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 347/688] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:44.797364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2a912-f524-429a-9977-3dc93a67b54a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.900690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 342/689] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:44.902271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952ca20-7e61-4e47-98d7-fadcc16dc6eb None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.911280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:44.917901 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2a912-f524-429a-9977-3dc93a67b54a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:44.955355 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ffd9e-733a-4728-a169-7017eee470cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:44.956026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ffd9e-733a-4728-a169-7017eee470cf None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:44.956305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ffd9e-733a-4728-a169-7017eee470cf None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:44.956577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ffd9e-733a-4728-a169-7017eee470cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:44.957523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ffd9e-733a-4728-a169-7017eee470cf None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=817b1a026f4b443daea514338ad8531a) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:44.957835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ffd9e-733a-4728-a169-7017eee470cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:44.959488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ffd9e-733a-4728-a169-7017eee470cf None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:44.966666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 348/690] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:44 2022] GET /identity/v3/role_assignments?user.id=817b1a026f4b443daea514338ad8531a&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:44.967520 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ffd9e-733a-4728-a169-7017eee470cf None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:44.988709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:45.015706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952ca20-7e61-4e47-98d7-fadcc16dc6eb None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:45.053610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa0791-8fd6-4b7b-8ed0-41363743cb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:45.054773 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa0791-8fd6-4b7b-8ed0-41363743cb93 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.055298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa0791-8fd6-4b7b-8ed0-41363743cb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.055858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa0791-8fd6-4b7b-8ed0-41363743cb93 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.056956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa0791-8fd6-4b7b-8ed0-41363743cb93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:45.057446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa0791-8fd6-4b7b-8ed0-41363743cb93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:45.059328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa0791-8fd6-4b7b-8ed0-41363743cb93 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:45.073276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 343/691] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:45.080058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ffd9e-733a-4728-a169-7017eee470cf None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:45.081431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa0791-8fd6-4b7b-8ed0-41363743cb93 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:45.123217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f5c4-d170-4825-9d2e-e2d0fd77933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:45.124072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f5c4-d170-4825-9d2e-e2d0fd77933c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.125501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f5c4-d170-4825-9d2e-e2d0fd77933c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.125785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f5c4-d170-4825-9d2e-e2d0fd77933c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.147558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516f5c4-d170-4825-9d2e-e2d0fd77933c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:45.149054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516f5c4-d170-4825-9d2e-e2d0fd77933c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-b516f5c4-d170-4825-9d2e-e2d0fd77933c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:45.181337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:45.182515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:45.183860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:45.183860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:45.183860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:45.183860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:45.183860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 349/692] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:45.188906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c9916-6cb1-4466-84f1-39c73ca4cb2f None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.189197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c9916-6cb1-4466-84f1-39c73ca4cb2f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.189459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c9916-6cb1-4466-84f1-39c73ca4cb2f None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.191275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 350/693] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:45.192904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c9916-6cb1-4466-84f1-39c73ca4cb2f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:45.206606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:45.247497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa0791-8fd6-4b7b-8ed0-41363743cb93 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:45.280727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:45.288311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee46b79-a679-45a7-9070-38c2265f235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:45.289133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee46b79-a679-45a7-9070-38c2265f235a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.289427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee46b79-a679-45a7-9070-38c2265f235a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.289698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee46b79-a679-45a7-9070-38c2265f235a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.299952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee46b79-a679-45a7-9070-38c2265f235a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:45.301397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee46b79-a679-45a7-9070-38c2265f235a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:45.316570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c9916-6cb1-4466-84f1-39c73ca4cb2f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-0ee46b79-a679-45a7-9070-38c2265f235a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:45.323358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:45.324299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:45.325232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 344/694] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 4 05:41:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:45.326294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465afc0-89f2-4a74-ad3b-e9d5c2031df2 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.328487 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465afc0-89f2-4a74-ad3b-e9d5c2031df2 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.328487 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465afc0-89f2-4a74-ad3b-e9d5c2031df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.344361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-5465afc0-89f2-4a74-ad3b-e9d5c2031df2 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:45.345590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5465afc0-89f2-4a74-ad3b-e9d5c2031df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:45.356521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce110f-162c-4e14-8e07-5341de6b2e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:45.357293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce110f-162c-4e14-8e07-5341de6b2e5c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.357570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce110f-162c-4e14-8e07-5341de6b2e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.357840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce110f-162c-4e14-8e07-5341de6b2e5c None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.358558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce110f-162c-4e14-8e07-5341de6b2e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:45.358855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce110f-162c-4e14-8e07-5341de6b2e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:45.360479 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce110f-162c-4e14-8e07-5341de6b2e5c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:45.373738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 351/695] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:45.374711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce110f-162c-4e14-8e07-5341de6b2e5c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:45.419197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 345/696] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:45.420087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465afc0-89f2-4a74-ad3b-e9d5c2031df2 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:45.492849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:45.505114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce110f-162c-4e14-8e07-5341de6b2e5c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:45.542120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:45.548769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cc221-270f-4b30-a4c1-f0ca51b2463e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:45.549456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cc221-270f-4b30-a4c1-f0ca51b2463e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.549738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cc221-270f-4b30-a4c1-f0ca51b2463e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.550011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cc221-270f-4b30-a4c1-f0ca51b2463e None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.550942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465afc0-89f2-4a74-ad3b-e9d5c2031df2 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:45.551061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cc221-270f-4b30-a4c1-f0ca51b2463e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:45.551388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cc221-270f-4b30-a4c1-f0ca51b2463e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:45.553643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cc221-270f-4b30-a4c1-f0ca51b2463e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:45.576978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 352/697] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 4 05:41:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:45.577988 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cc221-270f-4b30-a4c1-f0ca51b2463e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:45.598924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0d59e-ebd4-4510-ba4c-555c41952fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:45.600044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0d59e-ebd4-4510-ba4c-555c41952fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.600466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0d59e-ebd4-4510-ba4c-555c41952fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.601394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0d59e-ebd4-4510-ba4c-555c41952fd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.611428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0d59e-ebd4-4510-ba4c-555c41952fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:45.613357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0d59e-ebd4-4510-ba4c-555c41952fd6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-e5b0d59e-ebd4-4510-ba4c-555c41952fd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:45.635832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:45.636971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:45.638053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:45.638053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:45.638053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:45.638053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:45.638053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:45.638053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:45.638053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:45.638053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:45.640930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 346/698] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:45.641944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0d59e-ebd4-4510-ba4c-555c41952fd6 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:45.698262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cc221-270f-4b30-a4c1-f0ca51b2463e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:45.746475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de79ac8-0f0e-4c40-b479-41fc9f8cf932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:45.748243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de79ac8-0f0e-4c40-b479-41fc9f8cf932 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.748821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de79ac8-0f0e-4c40-b479-41fc9f8cf932 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.749343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de79ac8-0f0e-4c40-b479-41fc9f8cf932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.751216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0d59e-ebd4-4510-ba4c-555c41952fd6 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:45.762415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de79ac8-0f0e-4c40-b479-41fc9f8cf932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:45.764414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de79ac8-0f0e-4c40-b479-41fc9f8cf932 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:45.768947 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:45.788984 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cc027-e515-405a-81c5-7147d2ad1987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-9de79ac8-0f0e-4c40-b479-41fc9f8cf932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:45.789702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:45.790788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:45.791922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:45.791922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:45.791922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:45.791922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:45.792238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cc027-e515-405a-81c5-7147d2ad1987 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.792371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cc027-e515-405a-81c5-7147d2ad1987 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.792746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cc027-e515-405a-81c5-7147d2ad1987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.794259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 353/699] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:45.795234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de79ac8-0f0e-4c40-b479-41fc9f8cf932 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:45.842359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:45.872942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cc027-e515-405a-81c5-7147d2ad1987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:45.875551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cc027-e515-405a-81c5-7147d2ad1987 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:45.905151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de79ac8-0f0e-4c40-b479-41fc9f8cf932 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:45.942410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b681a-ce86-4212-983a-2e54bb383d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:45.943074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b681a-ce86-4212-983a-2e54bb383d0e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.943396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b681a-ce86-4212-983a-2e54bb383d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.943674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b681a-ce86-4212-983a-2e54bb383d0e None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.944798 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b681a-ce86-4212-983a-2e54bb383d0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:45.945109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b681a-ce86-4212-983a-2e54bb383d0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:45.946561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b681a-ce86-4212-983a-2e54bb383d0e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:45.958557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 354/700] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:45.960561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b681a-ce86-4212-983a-2e54bb383d0e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:45.985154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 347/701] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:45.987553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce81ba-868b-454c-ad96-a710fa7656d7 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:45.988347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce81ba-868b-454c-ad96-a710fa7656d7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:45.988950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce81ba-868b-454c-ad96-a710fa7656d7 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:45.990652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 348/702] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:45.991857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce81ba-868b-454c-ad96-a710fa7656d7 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:46.012491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:46.066996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:46.070449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b681a-ce86-4212-983a-2e54bb383d0e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:46.099822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce81ba-868b-454c-ad96-a710fa7656d7 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:46.106166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2852d21-724f-4650-9447-58b3d5287622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:46.106828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2852d21-724f-4650-9447-58b3d5287622 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:46.107102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2852d21-724f-4650-9447-58b3d5287622 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:46.107400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2852d21-724f-4650-9447-58b3d5287622 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:46.108118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2852d21-724f-4650-9447-58b3d5287622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:46.108417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2852d21-724f-4650-9447-58b3d5287622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:46.109790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2852d21-724f-4650-9447-58b3d5287622 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:46.126266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 355/703] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:46.126266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2852d21-724f-4650-9447-58b3d5287622 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:46.143864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e9925-ac39-402a-992d-815e07a7dbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:46.145020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9925-ac39-402a-992d-815e07a7dbcc None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:46.145305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9925-ac39-402a-992d-815e07a7dbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:46.145713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9925-ac39-402a-992d-815e07a7dbcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:46.146714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e9925-ac39-402a-992d-815e07a7dbcc None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=655b3c5b98fa4d91b328d04d42fc263a) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:46.147275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e9925-ac39-402a-992d-815e07a7dbcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:46.149435 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e9925-ac39-402a-992d-815e07a7dbcc None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:46.156239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 349/704] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:45 2022] GET /identity/v3/role_assignments?user.id=655b3c5b98fa4d91b328d04d42fc263a&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:46.157481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c8586-1a41-4b30-bd11-204c0edb3d6a None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:46.157763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c8586-1a41-4b30-bd11-204c0edb3d6a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:46.158030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c8586-1a41-4b30-bd11-204c0edb3d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:46.177010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-718c8586-1a41-4b30-bd11-204c0edb3d6a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:46.178583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-718c8586-1a41-4b30-bd11-204c0edb3d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:46.254202 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2852d21-724f-4650-9447-58b3d5287622 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:46.256730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 350/705] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:46.257921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c8586-1a41-4b30-bd11-204c0edb3d6a None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:46.289574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:46.309269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e652ab-e27e-4172-b757-162e11d4bb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:46.310602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e652ab-e27e-4172-b757-162e11d4bb44 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:46.311051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e652ab-e27e-4172-b757-162e11d4bb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:46.311628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e652ab-e27e-4172-b757-162e11d4bb44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:46.326694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e652ab-e27e-4172-b757-162e11d4bb44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:46.329024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e652ab-e27e-4172-b757-162e11d4bb44 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-42e652ab-e27e-4172-b757-162e11d4bb44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:46.353883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:46.355030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:46.356008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:46.356008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:46.356008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:46.356008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:46.359604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 356/706] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:46.360705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e652ab-e27e-4172-b757-162e11d4bb44 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:46.380800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:46.408401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c8586-1a41-4b30-bd11-204c0edb3d6a None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:46.454298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b69cd9-5ebb-4ce2-9ca1-5cc637a4f830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:46.455420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b69cd9-5ebb-4ce2-9ca1-5cc637a4f830 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:46.455862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b69cd9-5ebb-4ce2-9ca1-5cc637a4f830 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:46.456139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b69cd9-5ebb-4ce2-9ca1-5cc637a4f830 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:46.456884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b69cd9-5ebb-4ce2-9ca1-5cc637a4f830 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:46.458341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b69cd9-5ebb-4ce2-9ca1-5cc637a4f830 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:46.482637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 351/707] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:41:46 2022] POST /identity/v3/users => generated 254 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:41:46.483492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b69cd9-5ebb-4ce2-9ca1-5cc637a4f830 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:46.490774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e652ab-e27e-4172-b757-162e11d4bb44 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:46.539384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863a28e-055c-4824-be60-9b5bd72ffb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:46.540703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863a28e-055c-4824-be60-9b5bd72ffb81 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:46.541196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863a28e-055c-4824-be60-9b5bd72ffb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:46.541571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863a28e-055c-4824-be60-9b5bd72ffb81 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:46.542359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863a28e-055c-4824-be60-9b5bd72ffb81 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:46.543884 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863a28e-055c-4824-be60-9b5bd72ffb81 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-9863a28e-055c-4824-be60-9b5bd72ffb81 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:46.558979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:46.560086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:46.561340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 357/708] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Jul 4 05:41:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:46.568780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863a28e-055c-4824-be60-9b5bd72ffb81 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:46.613215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:46.683359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:46.699695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b69cd9-5ebb-4ce2-9ca1-5cc637a4f830 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:46.724044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863a28e-055c-4824-be60-9b5bd72ffb81 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:46.746756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73c030-5292-4e1d-b5de-114e39df877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:46.747476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c030-5292-4e1d-b5de-114e39df877d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:46.747759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c030-5292-4e1d-b5de-114e39df877d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:46.748036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c030-5292-4e1d-b5de-114e39df877d None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:46.748768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73c030-5292-4e1d-b5de-114e39df877d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:46.749079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73c030-5292-4e1d-b5de-114e39df877d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:46.751298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73c030-5292-4e1d-b5de-114e39df877d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:46.768888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 352/709] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:46.769858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282230b-6bad-4cb6-b8ed-dddd2a3787be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:46.770625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282230b-6bad-4cb6-b8ed-dddd2a3787be None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:46.770966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282230b-6bad-4cb6-b8ed-dddd2a3787be None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:46.775292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282230b-6bad-4cb6-b8ed-dddd2a3787be None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:46.775292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282230b-6bad-4cb6-b8ed-dddd2a3787be None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:46.775292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282230b-6bad-4cb6-b8ed-dddd2a3787be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:46.775292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282230b-6bad-4cb6-b8ed-dddd2a3787be None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:46.776594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73c030-5292-4e1d-b5de-114e39df877d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:46.784076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 358/710] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 4 05:41:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:46.789188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282230b-6bad-4cb6-b8ed-dddd2a3787be None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:46.917641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73c030-5292-4e1d-b5de-114e39df877d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:46.926056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282230b-6bad-4cb6-b8ed-dddd2a3787be None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:46.947216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:46.955576 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150191c-1496-41a7-bfb5-3bbdfc9ca738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:46.956385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150191c-1496-41a7-bfb5-3bbdfc9ca738 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:46.956658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150191c-1496-41a7-bfb5-3bbdfc9ca738 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:46.956922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150191c-1496-41a7-bfb5-3bbdfc9ca738 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:46.964332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150191c-1496-41a7-bfb5-3bbdfc9ca738 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:46.965990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150191c-1496-41a7-bfb5-3bbdfc9ca738 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:46.967551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b4c5f-b8b0-4c56-bd1a-c6037a64b176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:46.968684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b4c5f-b8b0-4c56-bd1a-c6037a64b176 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:46.969163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b4c5f-b8b0-4c56-bd1a-c6037a64b176 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:46.969684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b4c5f-b8b0-4c56-bd1a-c6037a64b176 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:46.970593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b4c5f-b8b0-4c56-bd1a-c6037a64b176 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:46.971098 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b4c5f-b8b0-4c56-bd1a-c6037a64b176 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:46.972714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b4c5f-b8b0-4c56-bd1a-c6037a64b176 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:46.979582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 359/711] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 4 05:41:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-1150191c-1496-41a7-bfb5-3bbdfc9ca738 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:46.983344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:46.984245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:46.985124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:46.985124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:46.985124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:46.985124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:46.985124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:46.985124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:46.985124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:46.985124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:46.986513 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 353/712] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:46.987761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b4c5f-b8b0-4c56-bd1a-c6037a64b176 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:46.992094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:47.101104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b4c5f-b8b0-4c56-bd1a-c6037a64b176 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:47.138536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baac3ab-756e-46a2-a0cb-59577460ce9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:47.139582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baac3ab-756e-46a2-a0cb-59577460ce9b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:47.140040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baac3ab-756e-46a2-a0cb-59577460ce9b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:47.140510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baac3ab-756e-46a2-a0cb-59577460ce9b None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:47.141503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baac3ab-756e-46a2-a0cb-59577460ce9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:47.141983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baac3ab-756e-46a2-a0cb-59577460ce9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:47.143664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baac3ab-756e-46a2-a0cb-59577460ce9b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:47.157601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 360/713] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:47.163438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150191c-1496-41a7-bfb5-3bbdfc9ca738 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:47.279900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150191c-1496-41a7-bfb5-3bbdfc9ca738 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:47.314253 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1b595-5c4f-45de-b5b1-680eaa23282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:47.314926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1b595-5c4f-45de-b5b1-680eaa23282a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:47.315259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1b595-5c4f-45de-b5b1-680eaa23282a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:47.315544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1b595-5c4f-45de-b5b1-680eaa23282a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:47.316382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1b595-5c4f-45de-b5b1-680eaa23282a None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=2cb245e4300e4d5fbdd425e936f4ac77) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:47.316687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1b595-5c4f-45de-b5b1-680eaa23282a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:47.318120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1b595-5c4f-45de-b5b1-680eaa23282a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:47.331432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 354/714] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:47 2022] GET /identity/v3/role_assignments?user.id=2cb245e4300e4d5fbdd425e936f4ac77&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:47.348046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696adad-1931-4743-8527-032035af073d None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:47.349050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696adad-1931-4743-8527-032035af073d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:47.349327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696adad-1931-4743-8527-032035af073d None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:47.350965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 361/715] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:47.357188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c68c3-c94c-4dbb-8978-fe2b0956636b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:47.357488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c68c3-c94c-4dbb-8978-fe2b0956636b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:47.357745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c68c3-c94c-4dbb-8978-fe2b0956636b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:47.376508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-884c68c3-c94c-4dbb-8978-fe2b0956636b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:47.377826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-884c68c3-c94c-4dbb-8978-fe2b0956636b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:47.442277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:47.471131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 355/716] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:47.505728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696adad-1931-4743-8527-032035af073d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:47.607227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:47.654958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696adad-1931-4743-8527-032035af073d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:47.700880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900b854-c46f-43b8-9772-76b60d6ec118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:47.701698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900b854-c46f-43b8-9772-76b60d6ec118 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:47.701975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900b854-c46f-43b8-9772-76b60d6ec118 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:47.702242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900b854-c46f-43b8-9772-76b60d6ec118 None admin] PATH_INFO: `/v3/services/network` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:47.702899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900b854-c46f-43b8-9772-76b60d6ec118 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:47.704442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900b854-c46f-43b8-9772-76b60d6ec118 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-9900b854-c46f-43b8-9772-76b60d6ec118 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:47.719346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:47.720474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:47.721401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 362/717] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:47 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:47.731218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c68c3-c94c-4dbb-8978-fe2b0956636b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:47.859604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c68c3-c94c-4dbb-8978-fe2b0956636b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:47.904851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dffb6e-3ec2-4751-938e-af40425732ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:47.905543 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dffb6e-3ec2-4751-938e-af40425732ed None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:47.905827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dffb6e-3ec2-4751-938e-af40425732ed None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:47.906103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dffb6e-3ec2-4751-938e-af40425732ed None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:47.906819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dffb6e-3ec2-4751-938e-af40425732ed None admin] RBAC: Adding query filter params (name=network) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:47.907125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dffb6e-3ec2-4751-938e-af40425732ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:47.908533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dffb6e-3ec2-4751-938e-af40425732ed None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:47.913328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 356/718] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:41:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:47.921284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900b854-c46f-43b8-9772-76b60d6ec118 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:48.076197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900b854-c46f-43b8-9772-76b60d6ec118 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:48.118184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:48.127668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161dabe-1a94-4353-a0bc-d8d6ff21083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:48.128348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161dabe-1a94-4353-a0bc-d8d6ff21083c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:48.128628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161dabe-1a94-4353-a0bc-d8d6ff21083c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:48.128899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161dabe-1a94-4353-a0bc-d8d6ff21083c None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:48.129612 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a161dabe-1a94-4353-a0bc-d8d6ff21083c None admin] RBAC: Adding query filter params (type=network) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:48.129915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a161dabe-1a94-4353-a0bc-d8d6ff21083c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:48.135341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a161dabe-1a94-4353-a0bc-d8d6ff21083c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:48.140557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 363/719] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:41:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:48.607271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebff50-ba36-4b1f-8ac4-7ccd65578ab0 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:48.607792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebff50-ba36-4b1f-8ac4-7ccd65578ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:48.607792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebff50-ba36-4b1f-8ac4-7ccd65578ab0 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:48.609544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 357/720] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:48.615472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f191-8e3e-4a5d-a4f5-fac8326616eb None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:48.616026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f191-8e3e-4a5d-a4f5-fac8326616eb None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:48.616026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f191-8e3e-4a5d-a4f5-fac8326616eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:48.633854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-19e9f191-8e3e-4a5d-a4f5-fac8326616eb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:48.635266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-19e9f191-8e3e-4a5d-a4f5-fac8326616eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:48.720701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 364/721] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:48.768874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebff50-ba36-4b1f-8ac4-7ccd65578ab0 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:48.827911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:48.904972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebff50-ba36-4b1f-8ac4-7ccd65578ab0 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:48.945381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76d2b0-5260-4322-815a-e247ae5d424b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:48.946199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76d2b0-5260-4322-815a-e247ae5d424b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:48.946489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76d2b0-5260-4322-815a-e247ae5d424b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:48.946822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76d2b0-5260-4322-815a-e247ae5d424b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:48.963468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76d2b0-5260-4322-815a-e247ae5d424b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-fb76d2b0-5260-4322-815a-e247ae5d424b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:48.971267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:48.972207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:48.973068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 358/722] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:48.978564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9f191-8e3e-4a5d-a4f5-fac8326616eb None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:49.129811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9f191-8e3e-4a5d-a4f5-fac8326616eb None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:49.152941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:49.171953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea3b7-25f3-49ce-8e1b-8095b897bc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:49.172655 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea3b7-25f3-49ce-8e1b-8095b897bc73 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.172939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea3b7-25f3-49ce-8e1b-8095b897bc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.173212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea3b7-25f3-49ce-8e1b-8095b897bc73 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.173972 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fea3b7-25f3-49ce-8e1b-8095b897bc73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:49.174275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fea3b7-25f3-49ce-8e1b-8095b897bc73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:49.175692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fea3b7-25f3-49ce-8e1b-8095b897bc73 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:49.184967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 365/723] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:49.190466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76d2b0-5260-4322-815a-e247ae5d424b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:49.338142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76d2b0-5260-4322-815a-e247ae5d424b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:49.340469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:49.373202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe6089-8c75-49fc-8749-8cbd86bab525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:49.374196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe6089-8c75-49fc-8749-8cbd86bab525 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.374609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe6089-8c75-49fc-8749-8cbd86bab525 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.374974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe6089-8c75-49fc-8749-8cbd86bab525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.385791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe6089-8c75-49fc-8749-8cbd86bab525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:49.387262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe6089-8c75-49fc-8749-8cbd86bab525 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-affe6089-8c75-49fc-8749-8cbd86bab525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:49.405120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:49.406051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:49.407026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:49.407026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:49.407026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:49.407026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:49.411332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 359/724] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:49.416495 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea3b7-25f3-49ce-8e1b-8095b897bc73 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:49.418941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560b081-6a71-4f59-9f36-900f7c2de26f None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.419227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560b081-6a71-4f59-9f36-900f7c2de26f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.419525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560b081-6a71-4f59-9f36-900f7c2de26f None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.421152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 360/725] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:41:49.426385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16ea0d-39a0-42f2-87d0-2c4bb9b2fa6f None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.426665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16ea0d-39a0-42f2-87d0-2c4bb9b2fa6f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.426934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16ea0d-39a0-42f2-87d0-2c4bb9b2fa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.449330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-cc16ea0d-39a0-42f2-87d0-2c4bb9b2fa6f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:49.450514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cc16ea0d-39a0-42f2-87d0-2c4bb9b2fa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:49.539494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 361/726] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 05:41:49.558852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16ea0d-39a0-42f2-87d0-2c4bb9b2fa6f None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:49.653626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea3b7-25f3-49ce-8e1b-8095b897bc73 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:49.724790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a04ee0-edeb-4717-a256-c3691cd0c0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:49.725468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a04ee0-edeb-4717-a256-c3691cd0c0dd None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.725742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a04ee0-edeb-4717-a256-c3691cd0c0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.726013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a04ee0-edeb-4717-a256-c3691cd0c0dd None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.726247 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:49.726743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a04ee0-edeb-4717-a256-c3691cd0c0dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:49.727038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a04ee0-edeb-4717-a256-c3691cd0c0dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:49.728702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a04ee0-edeb-4717-a256-c3691cd0c0dd None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:49.741234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16ea0d-39a0-42f2-87d0-2c4bb9b2fa6f None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:49.749353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 366/727] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:49.750713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f12f4-fab6-4f5c-9a0f-873b57b3c23b None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.751000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f12f4-fab6-4f5c-9a0f-873b57b3c23b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.751325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f12f4-fab6-4f5c-9a0f-873b57b3c23b None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.756698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 367/728] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:49.757616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f12f4-fab6-4f5c-9a0f-873b57b3c23b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:49.782987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9920c-0e6c-406d-a345-4dccbd7d1414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:49.784118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9920c-0e6c-406d-a345-4dccbd7d1414 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.784630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9920c-0e6c-406d-a345-4dccbd7d1414 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.784999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9920c-0e6c-406d-a345-4dccbd7d1414 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.796532 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9920c-0e6c-406d-a345-4dccbd7d1414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-c0a9920c-0e6c-406d-a345-4dccbd7d1414 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:49.806392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:49.807453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:49.808289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:49.810606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 362/729] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:49.811773 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04285bee-1fc8-48a7-a612-3cdaa79860ea None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.812062 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04285bee-1fc8-48a7-a612-3cdaa79860ea None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.812341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04285bee-1fc8-48a7-a612-3cdaa79860ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.815957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:49.831717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-04285bee-1fc8-48a7-a612-3cdaa79860ea None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:49.832914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-04285bee-1fc8-48a7-a612-3cdaa79860ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:49.875894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f12f4-fab6-4f5c-9a0f-873b57b3c23b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:49.903591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 363/730] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:49.905160 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c4766-d452-4293-8b97-e3350012b30e None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.905451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c4766-d452-4293-8b97-e3350012b30e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.905788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c4766-d452-4293-8b97-e3350012b30e None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.911426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 364/731] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:49.911426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c4766-d452-4293-8b97-e3350012b30e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:49.922334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5dfd8-a799-40a7-8945-add44d9ac18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:49.923606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5dfd8-a799-40a7-8945-add44d9ac18e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.924159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5dfd8-a799-40a7-8945-add44d9ac18e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.924674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5dfd8-a799-40a7-8945-add44d9ac18e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.933200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5dfd8-a799-40a7-8945-add44d9ac18e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:49.934887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5dfd8-a799-40a7-8945-add44d9ac18e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-9ac5dfd8-a799-40a7-8945-add44d9ac18e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:49.951499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:49.952444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:49.953385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:49.953385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:49.953385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:49.953385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:49.953385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:49.953385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:49.953385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:49.953385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:49.953385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:41:49.953385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:49.953923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 368/732] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:49.955030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aa05b-732b-41b7-9a6e-81da57852718 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:49.955349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aa05b-732b-41b7-9a6e-81da57852718 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:49.955624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aa05b-732b-41b7-9a6e-81da57852718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:49.972872 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-856aa05b-732b-41b7-9a6e-81da57852718 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:49.974060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-856aa05b-732b-41b7-9a6e-81da57852718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:49.992880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:50.016840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c4766-d452-4293-8b97-e3350012b30e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:50.053740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06201b1-68a8-444c-b2e2-bf0425048cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:50.054454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06201b1-68a8-444c-b2e2-bf0425048cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:50.054736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06201b1-68a8-444c-b2e2-bf0425048cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:50.055170 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 369/733] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:50.056046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856aa05b-732b-41b7-9a6e-81da57852718 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:50.063454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06201b1-68a8-444c-b2e2-bf0425048cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:50.064964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06201b1-68a8-444c-b2e2-bf0425048cc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:50.065471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06201b1-68a8-444c-b2e2-bf0425048cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:50.067544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06201b1-68a8-444c-b2e2-bf0425048cc2 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:50.079362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 365/734] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:50.080788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06201b1-68a8-444c-b2e2-bf0425048cc2 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:50.092612 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:50.165897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856aa05b-732b-41b7-9a6e-81da57852718 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:50.188848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06201b1-68a8-444c-b2e2-bf0425048cc2 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:50.200784 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a538c-bf89-4431-a205-4fd54e9d5422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:50.201462 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a538c-bf89-4431-a205-4fd54e9d5422 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:50.201743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a538c-bf89-4431-a205-4fd54e9d5422 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:50.202017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a538c-bf89-4431-a205-4fd54e9d5422 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:50.202735 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a538c-bf89-4431-a205-4fd54e9d5422 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:50.204657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a538c-bf89-4431-a205-4fd54e9d5422 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:50.227327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 370/735] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:41:50 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:41:50.227327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a538c-bf89-4431-a205-4fd54e9d5422 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:50.259781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6104c6-760f-4fd3-8cb1-c7f3f485df31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:50.260645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6104c6-760f-4fd3-8cb1-c7f3f485df31 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:50.260929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6104c6-760f-4fd3-8cb1-c7f3f485df31 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:50.261205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6104c6-760f-4fd3-8cb1-c7f3f485df31 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:50.262249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6104c6-760f-4fd3-8cb1-c7f3f485df31 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:50.262948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6104c6-760f-4fd3-8cb1-c7f3f485df31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:50.264929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6104c6-760f-4fd3-8cb1-c7f3f485df31 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:50.267995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:50.295678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 366/736] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 4 05:41:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:50.296679 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6104c6-760f-4fd3-8cb1-c7f3f485df31 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:50.350701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:50.361359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a538c-bf89-4431-a205-4fd54e9d5422 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:50.401442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6eefbc-76ce-4707-aa28-12d8890c47e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:50.402362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eefbc-76ce-4707-aa28-12d8890c47e3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:50.402702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eefbc-76ce-4707-aa28-12d8890c47e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:50.403037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eefbc-76ce-4707-aa28-12d8890c47e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:50.415492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6eefbc-76ce-4707-aa28-12d8890c47e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-ed6eefbc-76ce-4707-aa28-12d8890c47e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:50.424998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:50.425933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:50.426851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 371/737] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:50.427597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6eefbc-76ce-4707-aa28-12d8890c47e3 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:50.440592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6104c6-760f-4fd3-8cb1-c7f3f485df31 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:50.482709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda9ef-4361-4b34-b40b-24b709633594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:50.483633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda9ef-4361-4b34-b40b-24b709633594 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:50.483850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda9ef-4361-4b34-b40b-24b709633594 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:50.484122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda9ef-4361-4b34-b40b-24b709633594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:50.495194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda9ef-4361-4b34-b40b-24b709633594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:50.496694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda9ef-4361-4b34-b40b-24b709633594 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-51fda9ef-4361-4b34-b40b-24b709633594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:50.515034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:50.516002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:50.516886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:50.516886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:50.516886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:50.516886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:50.516886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 367/738] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:50.518392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda9ef-4361-4b34-b40b-24b709633594 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:50.557941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6eefbc-76ce-4707-aa28-12d8890c47e3 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:50.569579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:50.601675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690670b-7e42-4b81-a756-b7393fb0f052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:50.602332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690670b-7e42-4b81-a756-b7393fb0f052 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:50.602610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690670b-7e42-4b81-a756-b7393fb0f052 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:50.602878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690670b-7e42-4b81-a756-b7393fb0f052 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:50.632226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda9ef-4361-4b34-b40b-24b709633594 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:50.636120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:50.675584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfad4b-b204-4390-823d-612248c270c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:50.676682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfad4b-b204-4390-823d-612248c270c0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:50.677235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfad4b-b204-4390-823d-612248c270c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:50.677709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfad4b-b204-4390-823d-612248c270c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:50.678662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfad4b-b204-4390-823d-612248c270c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:50.679167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfad4b-b204-4390-823d-612248c270c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:50.680846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfad4b-b204-4390-823d-612248c270c0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:50.690685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 368/739] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:50.692049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfad4b-b204-4390-823d-612248c270c0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:50.698993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690670b-7e42-4b81-a756-b7393fb0f052 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:50.700498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690670b-7e42-4b81-a756-b7393fb0f052 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:50.799115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfad4b-b204-4390-823d-612248c270c0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:50.806211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:50.823796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 372/740] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:50 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 05:41:50.824648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690670b-7e42-4b81-a756-b7393fb0f052 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:50.838521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1f34f-5ff8-4bcc-a495-5f3e9b8971d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:50.839189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f34f-5ff8-4bcc-a495-5f3e9b8971d2 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:50.839495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f34f-5ff8-4bcc-a495-5f3e9b8971d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:50.839761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f34f-5ff8-4bcc-a495-5f3e9b8971d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:50.840510 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1f34f-5ff8-4bcc-a495-5f3e9b8971d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:50.840808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1f34f-5ff8-4bcc-a495-5f3e9b8971d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:50.842175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1f34f-5ff8-4bcc-a495-5f3e9b8971d2 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:50.857662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 369/741] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:50.857662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43f013-6622-46b0-8c97-f6c3c6d3aa81 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:50.857662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43f013-6622-46b0-8c97-f6c3c6d3aa81 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:50.857662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43f013-6622-46b0-8c97-f6c3c6d3aa81 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:50.857662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 370/742] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:50.857662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43f013-6622-46b0-8c97-f6c3c6d3aa81 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:50.888412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:50.936581 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690670b-7e42-4b81-a756-b7393fb0f052 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:50.966596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43f013-6622-46b0-8c97-f6c3c6d3aa81 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:50.972938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fa210-716f-4aa2-9453-e64646222114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:50.973755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa210-716f-4aa2-9453-e64646222114 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:50.974034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa210-716f-4aa2-9453-e64646222114 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:50.974307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa210-716f-4aa2-9453-e64646222114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:50.985107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171fa210-716f-4aa2-9453-e64646222114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:50.986594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171fa210-716f-4aa2-9453-e64646222114 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.001797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aeb924-96ec-47bd-a7ea-b9eba1e966a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:51.002601 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeb924-96ec-47bd-a7ea-b9eba1e966a4 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.002882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeb924-96ec-47bd-a7ea-b9eba1e966a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.003152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeb924-96ec-47bd-a7ea-b9eba1e966a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-171fa210-716f-4aa2-9453-e64646222114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:51.005334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:51.006239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:51.007137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:51.007137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:51.007137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:51.007137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:51.007137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 373/743] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:51.007913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fa210-716f-4aa2-9453-e64646222114 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:51.020677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeb924-96ec-47bd-a7ea-b9eba1e966a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:51.022148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeb924-96ec-47bd-a7ea-b9eba1e966a4 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-25aeb924-96ec-47bd-a7ea-b9eba1e966a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:51.039107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:51.040055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:51.041165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:51.041165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:51.041165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:51.041165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:51.041165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 371/744] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:51.041960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704382c4-54de-4a27-bc4b-75fa3cf7787c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.042032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704382c4-54de-4a27-bc4b-75fa3cf7787c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.042315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704382c4-54de-4a27-bc4b-75fa3cf7787c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.061259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-704382c4-54de-4a27-bc4b-75fa3cf7787c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:51.062456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-704382c4-54de-4a27-bc4b-75fa3cf7787c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:51.077746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:51.124147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fa210-716f-4aa2-9453-e64646222114 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:51.132545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:51.136797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 372/745] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 05:41:51.138148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704382c4-54de-4a27-bc4b-75fa3cf7787c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:51.161130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1167a1-39b3-4f4e-861d-29742ea6d723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:51.162102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1167a1-39b3-4f4e-861d-29742ea6d723 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.162376 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1167a1-39b3-4f4e-861d-29742ea6d723 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.162642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1167a1-39b3-4f4e-861d-29742ea6d723 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.170590 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1167a1-39b3-4f4e-861d-29742ea6d723 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:51.172070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1167a1-39b3-4f4e-861d-29742ea6d723 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-3a1167a1-39b3-4f4e-861d-29742ea6d723 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:51.187110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:51.188059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 374/746] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 4 05:41:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:51.189102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1167a1-39b3-4f4e-861d-29742ea6d723 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:51.249553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704382c4-54de-4a27-bc4b-75fa3cf7787c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:51.285473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b1eba-264d-42bb-9d1f-679f508ef87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:51.286150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b1eba-264d-42bb-9d1f-679f508ef87b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.286432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b1eba-264d-42bb-9d1f-679f508ef87b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.286708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b1eba-264d-42bb-9d1f-679f508ef87b None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.287472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b1eba-264d-42bb-9d1f-679f508ef87b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:51.287785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b1eba-264d-42bb-9d1f-679f508ef87b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:51.289184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b1eba-264d-42bb-9d1f-679f508ef87b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.294299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1167a1-39b3-4f4e-861d-29742ea6d723 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:51.305232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 373/747] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:51.305232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b1eba-264d-42bb-9d1f-679f508ef87b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:51.316025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:51.334891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be747f5c-2ba5-4990-bcbc-20df41db49bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:51.335635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be747f5c-2ba5-4990-bcbc-20df41db49bd None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.335918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be747f5c-2ba5-4990-bcbc-20df41db49bd None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.336197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be747f5c-2ba5-4990-bcbc-20df41db49bd None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.336921 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be747f5c-2ba5-4990-bcbc-20df41db49bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:51.337241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be747f5c-2ba5-4990-bcbc-20df41db49bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:51.338695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be747f5c-2ba5-4990-bcbc-20df41db49bd None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.350249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 375/748] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:51.351050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be747f5c-2ba5-4990-bcbc-20df41db49bd None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:51.383888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:51.416123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b1eba-264d-42bb-9d1f-679f508ef87b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:51.452158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4a6fc-866e-4d8f-b0a8-e1480d0c355e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:51.452761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4a6fc-866e-4d8f-b0a8-e1480d0c355e None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.453033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4a6fc-866e-4d8f-b0a8-e1480d0c355e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.453307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4a6fc-866e-4d8f-b0a8-e1480d0c355e None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.454034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4a6fc-866e-4d8f-b0a8-e1480d0c355e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:51.455442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4a6fc-866e-4d8f-b0a8-e1480d0c355e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.472118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 374/749] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:41:51 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:41:51.473207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4a6fc-866e-4d8f-b0a8-e1480d0c355e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:51.486473 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be747f5c-2ba5-4990-bcbc-20df41db49bd None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:51.551225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdab934-d976-445b-b938-5ea9ed7e59a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:51.551951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdab934-d976-445b-b938-5ea9ed7e59a6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.552233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdab934-d976-445b-b938-5ea9ed7e59a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.552504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdab934-d976-445b-b938-5ea9ed7e59a6 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.553357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdab934-d976-445b-b938-5ea9ed7e59a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:51.553664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdab934-d976-445b-b938-5ea9ed7e59a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:51.555519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdab934-d976-445b-b938-5ea9ed7e59a6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.556721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:51.584371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 376/750] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 4 05:41:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:51.585569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdab934-d976-445b-b938-5ea9ed7e59a6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:51.611194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4a6fc-866e-4d8f-b0a8-e1480d0c355e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:51.663360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f31eb-b998-4f76-8a6f-0c7ddd19ff01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:51.664180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f31eb-b998-4f76-8a6f-0c7ddd19ff01 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.664695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f31eb-b998-4f76-8a6f-0c7ddd19ff01 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.664986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f31eb-b998-4f76-8a6f-0c7ddd19ff01 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.674559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f31eb-b998-4f76-8a6f-0c7ddd19ff01 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:51.676752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f31eb-b998-4f76-8a6f-0c7ddd19ff01 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.679196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-810f31eb-b998-4f76-8a6f-0c7ddd19ff01 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:51.693484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:51.694803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 375/751] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 4 05:41:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:51.696284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f31eb-b998-4f76-8a6f-0c7ddd19ff01 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:51.708010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdab934-d976-445b-b938-5ea9ed7e59a6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:51.753304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e50c5-c186-42cd-bd25-357f170c7724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:51.754119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e50c5-c186-42cd-bd25-357f170c7724 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.754397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e50c5-c186-42cd-bd25-357f170c7724 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.754669 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e50c5-c186-42cd-bd25-357f170c7724 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.765419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e50c5-c186-42cd-bd25-357f170c7724 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:51.767229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e50c5-c186-42cd-bd25-357f170c7724 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-026e50c5-c186-42cd-bd25-357f170c7724 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:51.785708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:51.786658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:51.788004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:51.788004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:51.788004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:51.788004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:51.788004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:51.788004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:51.788004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:51.788004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:51.788004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 377/752] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:51.788491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e50c5-c186-42cd-bd25-357f170c7724 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:51.810761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f31eb-b998-4f76-8a6f-0c7ddd19ff01 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:51.843959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:51.854952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4da5a-cabb-412f-83c1-532d0af77429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:51.855701 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4da5a-cabb-412f-83c1-532d0af77429 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.856125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4da5a-cabb-412f-83c1-532d0af77429 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.856414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4da5a-cabb-412f-83c1-532d0af77429 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.907674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e50c5-c186-42cd-bd25-357f170c7724 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:51.945341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b565a2-1433-4bec-95bb-ababbb1a0ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:51.946386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b565a2-1433-4bec-95bb-ababbb1a0ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:51.946919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b565a2-1433-4bec-95bb-ababbb1a0ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:51.947414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b565a2-1433-4bec-95bb-ababbb1a0ef8 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:51.947518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4da5a-cabb-412f-83c1-532d0af77429 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:51.948563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b565a2-1433-4bec-95bb-ababbb1a0ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:51.948937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4da5a-cabb-412f-83c1-532d0af77429 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.949290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b565a2-1433-4bec-95bb-ababbb1a0ef8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:51.950914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b565a2-1433-4bec-95bb-ababbb1a0ef8 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:51.961139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:51.971792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 378/753] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 4 05:41:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:51.972990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b565a2-1433-4bec-95bb-ababbb1a0ef8 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:52.068447 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 376/754] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:41:52.070035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4da5a-cabb-412f-83c1-532d0af77429 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:52.080313 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b565a2-1433-4bec-95bb-ababbb1a0ef8 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:52.099410 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:52.126455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8c1bb-1a81-47b1-87ab-85f8ba1da0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:52.127126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8c1bb-1a81-47b1-87ab-85f8ba1da0af None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.127432 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8c1bb-1a81-47b1-87ab-85f8ba1da0af None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.127704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8c1bb-1a81-47b1-87ab-85f8ba1da0af None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.128500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8c1bb-1a81-47b1-87ab-85f8ba1da0af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:52.128806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8c1bb-1a81-47b1-87ab-85f8ba1da0af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:52.130780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8c1bb-1a81-47b1-87ab-85f8ba1da0af None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:52.147200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 379/755] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:52.148076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8c1bb-1a81-47b1-87ab-85f8ba1da0af None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:52.182596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4da5a-cabb-412f-83c1-532d0af77429 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:52.224578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463642fd-0cda-494a-bdd4-99bdb0c1ff6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:52.225920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:52.226329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463642fd-0cda-494a-bdd4-99bdb0c1ff6c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.227532 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463642fd-0cda-494a-bdd4-99bdb0c1ff6c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.228064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463642fd-0cda-494a-bdd4-99bdb0c1ff6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.240518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463642fd-0cda-494a-bdd4-99bdb0c1ff6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:52.242277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463642fd-0cda-494a-bdd4-99bdb0c1ff6c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-463642fd-0cda-494a-bdd4-99bdb0c1ff6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:52.262001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:52.262985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:52.264348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:52.264348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:52.264348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:52.264348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:52.266595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 377/756] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:52.267626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463642fd-0cda-494a-bdd4-99bdb0c1ff6c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:52.272737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8c1bb-1a81-47b1-87ab-85f8ba1da0af None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:52.320730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571aebf7-86eb-47bf-92f4-24255d934372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:52.322120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aebf7-86eb-47bf-92f4-24255d934372 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.322602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aebf7-86eb-47bf-92f4-24255d934372 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.323206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aebf7-86eb-47bf-92f4-24255d934372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.335156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571aebf7-86eb-47bf-92f4-24255d934372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:52.336924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571aebf7-86eb-47bf-92f4-24255d934372 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-571aebf7-86eb-47bf-92f4-24255d934372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:52.355571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:52.356671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:52.357668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:52.357668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:52.357668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:52.357668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:52.359654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 380/757] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:52.360597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571aebf7-86eb-47bf-92f4-24255d934372 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:52.364018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:52.387001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463642fd-0cda-494a-bdd4-99bdb0c1ff6c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:52.421606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c27e-0144-4da6-ae71-4b59d8c1d2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:52.422278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c27e-0144-4da6-ae71-4b59d8c1d2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.422562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c27e-0144-4da6-ae71-4b59d8c1d2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.422841 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c27e-0144-4da6-ae71-4b59d8c1d2e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.423716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5c27e-0144-4da6-ae71-4b59d8c1d2e6 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=655b3c5b98fa4d91b328d04d42fc263a) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:52.424031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5c27e-0144-4da6-ae71-4b59d8c1d2e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:52.425480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5c27e-0144-4da6-ae71-4b59d8c1d2e6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:52.432105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 378/758] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:52 2022] GET /identity/v3/role_assignments?user.id=655b3c5b98fa4d91b328d04d42fc263a&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:52.432728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c27e-0144-4da6-ae71-4b59d8c1d2e6 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:52.512529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571aebf7-86eb-47bf-92f4-24255d934372 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:52.543679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:52.554268 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983c7df-18c3-4d2b-b176-21a5ed940012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:52.555060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983c7df-18c3-4d2b-b176-21a5ed940012 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.555489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983c7df-18c3-4d2b-b176-21a5ed940012 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.555863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983c7df-18c3-4d2b-b176-21a5ed940012 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.556750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983c7df-18c3-4d2b-b176-21a5ed940012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:52.557199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983c7df-18c3-4d2b-b176-21a5ed940012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:52.558776 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983c7df-18c3-4d2b-b176-21a5ed940012 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:52.564936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c27e-0144-4da6-ae71-4b59d8c1d2e6 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:52.575656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 381/759] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:52.579019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a672a-fa08-4021-84d0-01b8fc4c07b6 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.579448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a672a-fa08-4021-84d0-01b8fc4c07b6 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.579824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a672a-fa08-4021-84d0-01b8fc4c07b6 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.581638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 382/760] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:52.582604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a672a-fa08-4021-84d0-01b8fc4c07b6 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:52.614589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf6e0c-1643-4902-9fa9-faba0a9c33b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:52.615730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf6e0c-1643-4902-9fa9-faba0a9c33b8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.616461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf6e0c-1643-4902-9fa9-faba0a9c33b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.616891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf6e0c-1643-4902-9fa9-faba0a9c33b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.618020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf6e0c-1643-4902-9fa9-faba0a9c33b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:52.618468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf6e0c-1643-4902-9fa9-faba0a9c33b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:52.620080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf6e0c-1643-4902-9fa9-faba0a9c33b8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:52.634685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 379/761] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:52.636309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf1600-6a41-4d5e-9644-36a40c9635ed None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.636737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf1600-6a41-4d5e-9644-36a40c9635ed None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.637150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf1600-6a41-4d5e-9644-36a40c9635ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.651616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:52.654125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-5adf1600-6a41-4d5e-9644-36a40c9635ed None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:52.655520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5adf1600-6a41-4d5e-9644-36a40c9635ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:52.707646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a672a-fa08-4021-84d0-01b8fc4c07b6 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:52.736355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 380/762] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:41:52.737564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf1600-6a41-4d5e-9644-36a40c9635ed None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:52.755135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f02349-29de-475f-a6f1-5ac969fc2a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:52.755985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f02349-29de-475f-a6f1-5ac969fc2a34 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.756267 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f02349-29de-475f-a6f1-5ac969fc2a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.756543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f02349-29de-475f-a6f1-5ac969fc2a34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.765920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f02349-29de-475f-a6f1-5ac969fc2a34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:52.767371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f02349-29de-475f-a6f1-5ac969fc2a34 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-76f02349-29de-475f-a6f1-5ac969fc2a34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:52.782459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:52.783434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:52.784366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:52.784366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:52.784366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:52.784366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:52.784366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:52.784366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:52.784366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:52.784366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:52.784366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 383/763] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:52.785387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f02349-29de-475f-a6f1-5ac969fc2a34 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:52.819572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:52.870617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf1600-6a41-4d5e-9644-36a40c9635ed None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:52.905837 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114e90b-d4d4-4740-bcb7-9aad25391cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:52.906656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114e90b-d4d4-4740-bcb7-9aad25391cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.906944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114e90b-d4d4-4740-bcb7-9aad25391cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.907208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114e90b-d4d4-4740-bcb7-9aad25391cd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.909327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:52.915083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114e90b-d4d4-4740-bcb7-9aad25391cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:52.916985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114e90b-d4d4-4740-bcb7-9aad25391cd4 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:52.918983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f02349-29de-475f-a6f1-5ac969fc2a34 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-d114e90b-d4d4-4740-bcb7-9aad25391cd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:52.932404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:52.933554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:52.934681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:52.934681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:52.934681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:52.934681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:52.934681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:52.934681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:52.934681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:52.934681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:52.939346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 381/764] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:52.940226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114e90b-d4d4-4740-bcb7-9aad25391cd4 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:52.962991 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482aa9-7634-40cc-af00-b8d69c92f6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:52.964513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482aa9-7634-40cc-af00-b8d69c92f6ec None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:52.964935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482aa9-7634-40cc-af00-b8d69c92f6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:52.965348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482aa9-7634-40cc-af00-b8d69c92f6ec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:52.966695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d482aa9-7634-40cc-af00-b8d69c92f6ec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:52.968550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d482aa9-7634-40cc-af00-b8d69c92f6ec None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-2d482aa9-7634-40cc-af00-b8d69c92f6ec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:52.985861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:52.987055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 05:41:52.988226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:52.991801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 384/765] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Jul 4 05:41:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:52.992738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482aa9-7634-40cc-af00-b8d69c92f6ec None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:53.062438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114e90b-d4d4-4740-bcb7-9aad25391cd4 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:53.096638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:53.101519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9afa5-516b-44c6-8e0d-56b1596264b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:53.102405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9afa5-516b-44c6-8e0d-56b1596264b1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:53.102856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9afa5-516b-44c6-8e0d-56b1596264b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:53.103366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9afa5-516b-44c6-8e0d-56b1596264b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:53.104323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482aa9-7634-40cc-af00-b8d69c92f6ec None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:53.104807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9afa5-516b-44c6-8e0d-56b1596264b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:53.105286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9afa5-516b-44c6-8e0d-56b1596264b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:53.107632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9afa5-516b-44c6-8e0d-56b1596264b1 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:53.122424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 382/766] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:53.123780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9afa5-516b-44c6-8e0d-56b1596264b1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:53.143984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf390f8d-cc78-47f7-9d64-af7a721ab89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:53.144806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf390f8d-cc78-47f7-9d64-af7a721ab89d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:53.145210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf390f8d-cc78-47f7-9d64-af7a721ab89d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:53.145596 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf390f8d-cc78-47f7-9d64-af7a721ab89d None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:53.146487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf390f8d-cc78-47f7-9d64-af7a721ab89d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:53.146892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf390f8d-cc78-47f7-9d64-af7a721ab89d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:53.148483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf390f8d-cc78-47f7-9d64-af7a721ab89d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:53.161466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 385/767] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:53.162444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf390f8d-cc78-47f7-9d64-af7a721ab89d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:53.187587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:53.238116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9afa5-516b-44c6-8e0d-56b1596264b1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:53.266647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf390f8d-cc78-47f7-9d64-af7a721ab89d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:53.275653 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb895f5-7089-41ec-85aa-877b27b87a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:53.276584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb895f5-7089-41ec-85aa-877b27b87a34 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:53.277081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb895f5-7089-41ec-85aa-877b27b87a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:53.277496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb895f5-7089-41ec-85aa-877b27b87a34 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:53.278445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb895f5-7089-41ec-85aa-877b27b87a34 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:53.278964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb895f5-7089-41ec-85aa-877b27b87a34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:53.280578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb895f5-7089-41ec-85aa-877b27b87a34 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:53.285944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 383/768] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 4 05:41:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:53.286949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb895f5-7089-41ec-85aa-877b27b87a34 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:53.307736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1d4e1-356e-490d-be11-37a47c71b0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:53.308645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1d4e1-356e-490d-be11-37a47c71b0fe None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:53.309058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1d4e1-356e-490d-be11-37a47c71b0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:53.309416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1d4e1-356e-490d-be11-37a47c71b0fe None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/2cb245e4300e4d5fbdd425e936f4ac77/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:53.311471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1d4e1-356e-490d-be11-37a47c71b0fe None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=2cb245e4300e4d5fbdd425e936f4ac77, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:53.313019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1d4e1-356e-490d-be11-37a47c71b0fe None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:53.330241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 386/769] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:41:53 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/2cb245e4300e4d5fbdd425e936f4ac77/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:53.331154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1d4e1-356e-490d-be11-37a47c71b0fe None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:53.348260 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:53.407230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb895f5-7089-41ec-85aa-877b27b87a34 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:53.427322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:53.440965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1d4e1-356e-490d-be11-37a47c71b0fe None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:53.447932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0404a-6684-4183-aa6f-1747ac268f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:53.448910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0404a-6684-4183-aa6f-1747ac268f4d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:53.449422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0404a-6684-4183-aa6f-1747ac268f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:53.449932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0404a-6684-4183-aa6f-1747ac268f4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:53.451020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0404a-6684-4183-aa6f-1747ac268f4d None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=884ed6ddb71647088b33dc9623143b9c) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:53.451578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0404a-6684-4183-aa6f-1747ac268f4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:53.453252 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0404a-6684-4183-aa6f-1747ac268f4d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:53.460190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 384/770] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:53 2022] GET /identity/v3/role_assignments?user.id=884ed6ddb71647088b33dc9623143b9c&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:53.482860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c7be6-e118-4e98-86ca-3997c9de3b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:53.483621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c7be6-e118-4e98-86ca-3997c9de3b97 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:53.483914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c7be6-e118-4e98-86ca-3997c9de3b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:53.484190 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c7be6-e118-4e98-86ca-3997c9de3b97 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:53.485086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c7be6-e118-4e98-86ca-3997c9de3b97 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:53.485404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c7be6-e118-4e98-86ca-3997c9de3b97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:53.486876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c7be6-e118-4e98-86ca-3997c9de3b97 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:53.493464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 387/771] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 4 05:41:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:53.499712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0404a-6684-4183-aa6f-1747ac268f4d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:53.623197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0404a-6684-4183-aa6f-1747ac268f4d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:53.661568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0156aa-9d3b-4c0b-a814-94fbffeb8009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:53.662250 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0156aa-9d3b-4c0b-a814-94fbffeb8009 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:53.662530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0156aa-9d3b-4c0b-a814-94fbffeb8009 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:53.662804 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0156aa-9d3b-4c0b-a814-94fbffeb8009 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:53.664223 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0156aa-9d3b-4c0b-a814-94fbffeb8009 None admin] RBAC: Adding query filter params (interface=public, service_id=7f5a4598aa224302a8d6c238479d476a, region_id=RegionOne) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:53.664535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0156aa-9d3b-4c0b-a814-94fbffeb8009 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:53.665987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0156aa-9d3b-4c0b-a814-94fbffeb8009 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:53.674418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 385/772] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 4 05:41:53 2022] GET /identity/v3/endpoints?service_id=7f5a4598aa224302a8d6c238479d476a&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:53.679909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c7be6-e118-4e98-86ca-3997c9de3b97 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:53.777363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:53.796056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c7be6-e118-4e98-86ca-3997c9de3b97 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:53.804335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829041a0-9b1c-4138-818f-3e33ce21a8b0 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:53.804651 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829041a0-9b1c-4138-818f-3e33ce21a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:53.804884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829041a0-9b1c-4138-818f-3e33ce21a8b0 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:53.807707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 386/773] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:53.811895 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43e0d6-cf41-4428-80bf-6dd37e448680 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:53.812182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43e0d6-cf41-4428-80bf-6dd37e448680 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:53.812456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43e0d6-cf41-4428-80bf-6dd37e448680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:53.831804 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-cc43e0d6-cf41-4428-80bf-6dd37e448680 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:53.832997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cc43e0d6-cf41-4428-80bf-6dd37e448680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:53.834949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12232e3b-0f16-4d73-add4-2713708909df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:53.835625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12232e3b-0f16-4d73-add4-2713708909df None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:53.835948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12232e3b-0f16-4d73-add4-2713708909df None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:53.836871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12232e3b-0f16-4d73-add4-2713708909df None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:53.836984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12232e3b-0f16-4d73-add4-2713708909df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:53.838349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12232e3b-0f16-4d73-add4-2713708909df None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:53.844178 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 388/774] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 4 05:41:53 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:53.906902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:53.946238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 387/775] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:41:53.994294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12232e3b-0f16-4d73-add4-2713708909df None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:54.102704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12232e3b-0f16-4d73-add4-2713708909df None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:54.141075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d34a8-924d-4c85-aa0f-51d75a60a3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:54.141902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d34a8-924d-4c85-aa0f-51d75a60a3bb None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:54.142181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d34a8-924d-4c85-aa0f-51d75a60a3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:54.142453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d34a8-924d-4c85-aa0f-51d75a60a3bb None admin] PATH_INFO: `/v3/services/network` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:54.143165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d34a8-924d-4c85-aa0f-51d75a60a3bb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:54.144621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d34a8-924d-4c85-aa0f-51d75a60a3bb None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-5c6d34a8-924d-4c85-aa0f-51d75a60a3bb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:54.157466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:54.158468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:54.159347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 389/776] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:53 2022] GET /identity/v3/services/network => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:54.164636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43e0d6-cf41-4428-80bf-6dd37e448680 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:54.291121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43e0d6-cf41-4428-80bf-6dd37e448680 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:54.334762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ceaab-9602-4476-a60c-41b82125e027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:54.334762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ceaab-9602-4476-a60c-41b82125e027 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:54.334762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ceaab-9602-4476-a60c-41b82125e027 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:54.334762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ceaab-9602-4476-a60c-41b82125e027 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:54.335191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ceaab-9602-4476-a60c-41b82125e027 None admin] RBAC: Adding query filter params (name=network) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:54.335191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ceaab-9602-4476-a60c-41b82125e027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:54.335191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ceaab-9602-4476-a60c-41b82125e027 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:54.343764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 388/777] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:41:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:54.349379 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d34a8-924d-4c85-aa0f-51d75a60a3bb None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:54.368460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:54.482924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d34a8-924d-4c85-aa0f-51d75a60a3bb None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:54.533008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65f3f2-f880-497a-b3b8-0c1c9aaece1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:54.533691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65f3f2-f880-497a-b3b8-0c1c9aaece1f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:54.533969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65f3f2-f880-497a-b3b8-0c1c9aaece1f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:54.534238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65f3f2-f880-497a-b3b8-0c1c9aaece1f None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:54.534951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65f3f2-f880-497a-b3b8-0c1c9aaece1f None admin] RBAC: Adding query filter params (type=network) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:54.535280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65f3f2-f880-497a-b3b8-0c1c9aaece1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:54.536719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65f3f2-f880-497a-b3b8-0c1c9aaece1f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:54.547723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 390/778] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:41:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:54.553206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ceaab-9602-4476-a60c-41b82125e027 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:54.629352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:54.704681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ceaab-9602-4476-a60c-41b82125e027 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:54.733698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06255d7-955c-4203-b038-e31c000f7abe None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:54.733964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06255d7-955c-4203-b038-e31c000f7abe None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:54.734150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06255d7-955c-4203-b038-e31c000f7abe None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:54.735715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 391/779] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:54.741769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e488392-4e4c-4bed-ad95-332e2ffaa8b1 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:54.742055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e488392-4e4c-4bed-ad95-332e2ffaa8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:54.742325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e488392-4e4c-4bed-ad95-332e2ffaa8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:54.754605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b9f4a-81af-4349-8a2a-9516d065ebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:54.755604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b9f4a-81af-4349-8a2a-9516d065ebab None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:54.756083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b9f4a-81af-4349-8a2a-9516d065ebab None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:54.756551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b9f4a-81af-4349-8a2a-9516d065ebab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:54.757541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b9f4a-81af-4349-8a2a-9516d065ebab None admin] RBAC: Adding query filter params (interface=public, service_id=df51660c169a4f7db4ed709dac800d3b, region_id=RegionOne) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:54.758067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b9f4a-81af-4349-8a2a-9516d065ebab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:54.759740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b9f4a-81af-4349-8a2a-9516d065ebab None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:54.765461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 389/780] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 4 05:41:54 2022] GET /identity/v3/endpoints?service_id=df51660c169a4f7db4ed709dac800d3b&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:54.771316 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-1e488392-4e4c-4bed-ad95-332e2ffaa8b1 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:54.771316 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1e488392-4e4c-4bed-ad95-332e2ffaa8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:54.780000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b9f4a-81af-4349-8a2a-9516d065ebab None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:54.848164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 392/781] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:41:54.849153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:54.872513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e488392-4e4c-4bed-ad95-332e2ffaa8b1 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:55.003322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b9f4a-81af-4349-8a2a-9516d065ebab None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:55.009883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:55.056300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e488392-4e4c-4bed-ad95-332e2ffaa8b1 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:55.067342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d62eb-ce8e-41a2-81ca-9f4dd8969b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:55.068025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d62eb-ce8e-41a2-81ca-9f4dd8969b4f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:55.068305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d62eb-ce8e-41a2-81ca-9f4dd8969b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:55.068572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d62eb-ce8e-41a2-81ca-9f4dd8969b4f None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:55.069609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d62eb-ce8e-41a2-81ca-9f4dd8969b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:55.070988 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d62eb-ce8e-41a2-81ca-9f4dd8969b4f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:55.081090 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 390/782] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 4 05:41:54 2022] GET /identity/v3/services => generated 772 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:55.115751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c485e63-cad9-4ed6-8ca6-712d9223ddda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:55.116574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485e63-cad9-4ed6-8ca6-712d9223ddda None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:55.116852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485e63-cad9-4ed6-8ca6-712d9223ddda None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:55.117126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485e63-cad9-4ed6-8ca6-712d9223ddda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:55.141608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c485e63-cad9-4ed6-8ca6-712d9223ddda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-3c485e63-cad9-4ed6-8ca6-712d9223ddda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:55.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:55.150729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:55.151994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 393/783] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:55.157169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d62eb-ce8e-41a2-81ca-9f4dd8969b4f None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:55.284848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d62eb-ce8e-41a2-81ca-9f4dd8969b4f None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:55.322405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ef533-1668-44d2-9442-cd14e4139c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:55.323089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ef533-1668-44d2-9442-cd14e4139c45 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:55.323404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ef533-1668-44d2-9442-cd14e4139c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:55.323679 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ef533-1668-44d2-9442-cd14e4139c45 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:55.324432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ef533-1668-44d2-9442-cd14e4139c45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:55.324737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ef533-1668-44d2-9442-cd14e4139c45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:55.326113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ef533-1668-44d2-9442-cd14e4139c45 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:55.335010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 391/784] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:55.341120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c485e63-cad9-4ed6-8ca6-712d9223ddda None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:55.400665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:55.441783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c485e63-cad9-4ed6-8ca6-712d9223ddda None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:55.483761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915f177-89b0-4aea-a3a9-faadf8182340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:55.484568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915f177-89b0-4aea-a3a9-faadf8182340 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:55.484847 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915f177-89b0-4aea-a3a9-faadf8182340 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:55.485114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915f177-89b0-4aea-a3a9-faadf8182340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:55.498920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915f177-89b0-4aea-a3a9-faadf8182340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:55.500451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915f177-89b0-4aea-a3a9-faadf8182340 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-0915f177-89b0-4aea-a3a9-faadf8182340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:55.529251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:55.530145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:55.531025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:55.531025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:55.531025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:55.531025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:55.536446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ef533-1668-44d2-9442-cd14e4139c45 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:55.537061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 394/785] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:55.594831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:55.679258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ef533-1668-44d2-9442-cd14e4139c45 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:55.726566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58568aae-d3f2-4052-90e3-329920701a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:55.727306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58568aae-d3f2-4052-90e3-329920701a4d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:55.727594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58568aae-d3f2-4052-90e3-329920701a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:55.727864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58568aae-d3f2-4052-90e3-329920701a4d None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:55.728631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58568aae-d3f2-4052-90e3-329920701a4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:55.728934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58568aae-d3f2-4052-90e3-329920701a4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:55.730320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58568aae-d3f2-4052-90e3-329920701a4d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:55.750631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 392/786] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:55.756829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915f177-89b0-4aea-a3a9-faadf8182340 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:55.938083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915f177-89b0-4aea-a3a9-faadf8182340 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:55.960756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:55.994913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d3ee6-43a1-4605-b267-dae8ed99aea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:55.996257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d3ee6-43a1-4605-b267-dae8ed99aea1 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:55.996734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d3ee6-43a1-4605-b267-dae8ed99aea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:55.997217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d3ee6-43a1-4605-b267-dae8ed99aea1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.010718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d3ee6-43a1-4605-b267-dae8ed99aea1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:56.013397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d3ee6-43a1-4605-b267-dae8ed99aea1 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-b52d3ee6-43a1-4605-b267-dae8ed99aea1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:56.040208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:56.041265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:56.042196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:56.042196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:56.042196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:56.042196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:56.042196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:56.042196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:56.042196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:56.042196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:56.042196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:41:56.042196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:56.043035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 395/787] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:56.052389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58568aae-d3f2-4052-90e3-329920701a4d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:56.092339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e4419-d217-4e2d-bd6f-a92e114bd28c None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.092656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e4419-d217-4e2d-bd6f-a92e114bd28c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.092895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e4419-d217-4e2d-bd6f-a92e114bd28c None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.094540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 396/788] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:56.099420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5b33e-6121-4bf5-830f-92ce95ecda11 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.099705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5b33e-6121-4bf5-830f-92ce95ecda11 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.099978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5b33e-6121-4bf5-830f-92ce95ecda11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.117465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-77d5b33e-6121-4bf5-830f-92ce95ecda11 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:56.118702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-77d5b33e-6121-4bf5-830f-92ce95ecda11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:56.188667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 397/789] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:41:56.189954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66911928-8a3a-41f1-85c5-34908f85b278 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.190238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66911928-8a3a-41f1-85c5-34908f85b278 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.190510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66911928-8a3a-41f1-85c5-34908f85b278 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.195874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 398/790] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:56.208166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2c44f-5f4c-40bc-8d9a-ec7bacfc5436 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.208668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2c44f-5f4c-40bc-8d9a-ec7bacfc5436 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.209142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2c44f-5f4c-40bc-8d9a-ec7bacfc5436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.228169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-53a2c44f-5f4c-40bc-8d9a-ec7bacfc5436 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:56.229652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-53a2c44f-5f4c-40bc-8d9a-ec7bacfc5436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:56.238528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:56.251006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58568aae-d3f2-4052-90e3-329920701a4d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:56.287093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc023119-efae-42f7-a68f-00583f3ed10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:56.288054 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc023119-efae-42f7-a68f-00583f3ed10c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.288588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc023119-efae-42f7-a68f-00583f3ed10c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.289007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc023119-efae-42f7-a68f-00583f3ed10c None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.290004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc023119-efae-42f7-a68f-00583f3ed10c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:56.290485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc023119-efae-42f7-a68f-00583f3ed10c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:56.292123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc023119-efae-42f7-a68f-00583f3ed10c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:56.316332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 393/791] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 4 05:41:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:56.317235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc023119-efae-42f7-a68f-00583f3ed10c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:56.318087 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 399/792] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:41:56.335151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2c44f-5f4c-40bc-8d9a-ec7bacfc5436 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:56.371972 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:56.444610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc023119-efae-42f7-a68f-00583f3ed10c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:56.455648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2c44f-5f4c-40bc-8d9a-ec7bacfc5436 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:56.483174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadda8e-c7ba-4877-87c7-520e27b3dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:56.484015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadda8e-c7ba-4877-87c7-520e27b3dccb None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.484292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadda8e-c7ba-4877-87c7-520e27b3dccb None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.484564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadda8e-c7ba-4877-87c7-520e27b3dccb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.493063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae8ee6-ae69-41f2-ba74-180eeb551a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:56.493949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae8ee6-ae69-41f2-ba74-180eeb551a0e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.494227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae8ee6-ae69-41f2-ba74-180eeb551a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.494499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae8ee6-ae69-41f2-ba74-180eeb551a0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.495429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadda8e-c7ba-4877-87c7-520e27b3dccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-2cadda8e-c7ba-4877-87c7-520e27b3dccb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:56.503086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:56.504076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:56.504942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 394/793] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:41:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:56.505649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadda8e-c7ba-4877-87c7-520e27b3dccb None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:56.511349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae8ee6-ae69-41f2-ba74-180eeb551a0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:56.514753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae8ee6-ae69-41f2-ba74-180eeb551a0e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:56.517640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-2fae8ee6-ae69-41f2-ba74-180eeb551a0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:56.532738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:56.533686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:56.534577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:56.534577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:56.534577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:56.534577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:56.534577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 400/794] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:41:56.535295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1c6f9-9dce-498f-9863-beac5cc9709c None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.535514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1c6f9-9dce-498f-9863-beac5cc9709c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.536020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1c6f9-9dce-498f-9863-beac5cc9709c None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.541402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 401/795] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:56.542361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1c6f9-9dce-498f-9863-beac5cc9709c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:56.625152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadda8e-c7ba-4877-87c7-520e27b3dccb None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:56.636816 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:56.656145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1c6f9-9dce-498f-9863-beac5cc9709c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:56.662028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573292bf-88af-424b-acc4-0fe12fecc9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:56.662837 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573292bf-88af-424b-acc4-0fe12fecc9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.663115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573292bf-88af-424b-acc4-0fe12fecc9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.663646 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573292bf-88af-424b-acc4-0fe12fecc9f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.673454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573292bf-88af-424b-acc4-0fe12fecc9f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-573292bf-88af-424b-acc4-0fe12fecc9f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:41:56.681173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:56.682092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:56.682955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 395/796] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:41:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:56.684244 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7324d36-95f0-447d-b589-f0178598e672 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.684244 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7324d36-95f0-447d-b589-f0178598e672 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.684244 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7324d36-95f0-447d-b589-f0178598e672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.697735 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b477e-5f4e-4314-9bee-ab076872af9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:56.698343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b477e-5f4e-4314-9bee-ab076872af9b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.698641 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b477e-5f4e-4314-9bee-ab076872af9b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.698893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b477e-5f4e-4314-9bee-ab076872af9b None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.699683 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b477e-5f4e-4314-9bee-ab076872af9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:56.699988 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b477e-5f4e-4314-9bee-ab076872af9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:56.701115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c7324d36-95f0-447d-b589-f0178598e672 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:56.701390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b477e-5f4e-4314-9bee-ab076872af9b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:56.702300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c7324d36-95f0-447d-b589-f0178598e672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:56.710099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 402/797] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:41:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:56.710099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b477e-5f4e-4314-9bee-ab076872af9b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:56.769035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:56.775512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 396/798] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:41:56.776698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7324d36-95f0-447d-b589-f0178598e672 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:56.831824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b477e-5f4e-4314-9bee-ab076872af9b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:56.871433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216930e1-332c-4b11-b9a3-0467ad267d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:56.872218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216930e1-332c-4b11-b9a3-0467ad267d98 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.872498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216930e1-332c-4b11-b9a3-0467ad267d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.872776 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216930e1-332c-4b11-b9a3-0467ad267d98 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.873501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216930e1-332c-4b11-b9a3-0467ad267d98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:56.873864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216930e1-332c-4b11-b9a3-0467ad267d98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:56.875523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216930e1-332c-4b11-b9a3-0467ad267d98 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:56.877659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7324d36-95f0-447d-b589-f0178598e672 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:56.892657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 403/799] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:56.892657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216930e1-332c-4b11-b9a3-0467ad267d98 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:56.899502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:56.919322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10942f-0d4b-4956-bd3a-bb7469b18cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:56.920008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10942f-0d4b-4956-bd3a-bb7469b18cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:56.920320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10942f-0d4b-4956-bd3a-bb7469b18cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:56.920570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10942f-0d4b-4956-bd3a-bb7469b18cf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:56.921284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10942f-0d4b-4956-bd3a-bb7469b18cf0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:56.921585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10942f-0d4b-4956-bd3a-bb7469b18cf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:56.922964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10942f-0d4b-4956-bd3a-bb7469b18cf0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:56.936371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 397/800] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:41:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:56.937273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10942f-0d4b-4956-bd3a-bb7469b18cf0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:56.995826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216930e1-332c-4b11-b9a3-0467ad267d98 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.000146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:57.031040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599b56-7d07-4721-8cf9-4e76c46aac8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.031905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599b56-7d07-4721-8cf9-4e76c46aac8b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.032187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599b56-7d07-4721-8cf9-4e76c46aac8b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.032463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599b56-7d07-4721-8cf9-4e76c46aac8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.045002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52599b56-7d07-4721-8cf9-4e76c46aac8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.046504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52599b56-7d07-4721-8cf9-4e76c46aac8b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-52599b56-7d07-4721-8cf9-4e76c46aac8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:57.066759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:57.068013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:57.068998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:57.068998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:57.068998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:57.068998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:57.068998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 404/801] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:57.069627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599b56-7d07-4721-8cf9-4e76c46aac8b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:57.133203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:57.165264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10942f-0d4b-4956-bd3a-bb7469b18cf0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.179202 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599b56-7d07-4721-8cf9-4e76c46aac8b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.199973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbea3cd-a0ce-4b3e-a331-5a91ec6d6cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.200791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbea3cd-a0ce-4b3e-a331-5a91ec6d6cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.201128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbea3cd-a0ce-4b3e-a331-5a91ec6d6cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.201401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbea3cd-a0ce-4b3e-a331-5a91ec6d6cb8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.202069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbea3cd-a0ce-4b3e-a331-5a91ec6d6cb8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.203467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbea3cd-a0ce-4b3e-a331-5a91ec6d6cb8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.218623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5830a08-c9dd-481f-bd49-8671d6de2ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-2dbea3cd-a0ce-4b3e-a331-5a91ec6d6cb8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:57.218959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:57.220010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:57.220960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 398/802] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Jul 4 05:41:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:57.221954 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbea3cd-a0ce-4b3e-a331-5a91ec6d6cb8 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:57.222821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5830a08-c9dd-481f-bd49-8671d6de2ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.223259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5830a08-c9dd-481f-bd49-8671d6de2ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.223699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5830a08-c9dd-481f-bd49-8671d6de2ae6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.231451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5830a08-c9dd-481f-bd49-8671d6de2ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.233117 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5830a08-c9dd-481f-bd49-8671d6de2ae6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-b5830a08-c9dd-481f-bd49-8671d6de2ae6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:57.247351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:57.248442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:57.249523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:57.249523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:57.249523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:57.249523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:57.249523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:57.249523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:57.249523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:57.249523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:57.251340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 405/803] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:57.252264 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5830a08-c9dd-481f-bd49-8671d6de2ae6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:57.328581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbea3cd-a0ce-4b3e-a331-5a91ec6d6cb8 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.352910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:57.359226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5830a08-c9dd-481f-bd49-8671d6de2ae6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.365145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4678aa-b050-4c28-a0fc-5f195723a5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.365959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4678aa-b050-4c28-a0fc-5f195723a5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.366237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4678aa-b050-4c28-a0fc-5f195723a5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.366505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4678aa-b050-4c28-a0fc-5f195723a5f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.377781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4678aa-b050-4c28-a0fc-5f195723a5f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.379380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:57.379508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4678aa-b050-4c28-a0fc-5f195723a5f1 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.397243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a9c9d-445e-42c1-a279-dfa5d88ee348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-af4678aa-b050-4c28-a0fc-5f195723a5f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:57.397818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:57.398647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:57.404686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:57.404686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:57.404686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:57.404686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:57.404907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a9c9d-445e-42c1-a279-dfa5d88ee348 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.404907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a9c9d-445e-42c1-a279-dfa5d88ee348 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.404907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a9c9d-445e-42c1-a279-dfa5d88ee348 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.404907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a9c9d-445e-42c1-a279-dfa5d88ee348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:57.405188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 399/804] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:57.405255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a9c9d-445e-42c1-a279-dfa5d88ee348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.405322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4678aa-b050-4c28-a0fc-5f195723a5f1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:57.405398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a9c9d-445e-42c1-a279-dfa5d88ee348 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.416076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 406/805] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:57.416716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a9c9d-445e-42c1-a279-dfa5d88ee348 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:57.519134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4678aa-b050-4c28-a0fc-5f195723a5f1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.522463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a9c9d-445e-42c1-a279-dfa5d88ee348 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.556474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb888e5-cb57-437f-9871-7198cbf21f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.557258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb888e5-cb57-437f-9871-7198cbf21f3e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.557576 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb888e5-cb57-437f-9871-7198cbf21f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.557918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb888e5-cb57-437f-9871-7198cbf21f3e None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.558692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb888e5-cb57-437f-9871-7198cbf21f3e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:57.559063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb888e5-cb57-437f-9871-7198cbf21f3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.559154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabc086-4c3d-4838-b11b-d339ff66037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.559881 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabc086-4c3d-4838-b11b-d339ff66037f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.560224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabc086-4c3d-4838-b11b-d339ff66037f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.560578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabc086-4c3d-4838-b11b-d339ff66037f None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.560772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb888e5-cb57-437f-9871-7198cbf21f3e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.561488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabc086-4c3d-4838-b11b-d339ff66037f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:57.561865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabc086-4c3d-4838-b11b-d339ff66037f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.563414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabc086-4c3d-4838-b11b-d339ff66037f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.571044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 400/806] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 4 05:41:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:57.571044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb888e5-cb57-437f-9871-7198cbf21f3e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:57.580940 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 407/807] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:57.581638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabc086-4c3d-4838-b11b-d339ff66037f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:57.619233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:57.632123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:57.687525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb888e5-cb57-437f-9871-7198cbf21f3e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.694574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabc086-4c3d-4838-b11b-d339ff66037f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.723835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07663a-5565-4c5f-b069-27f45b8cf1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.724608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07663a-5565-4c5f-b069-27f45b8cf1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.725012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07663a-5565-4c5f-b069-27f45b8cf1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.725295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07663a-5565-4c5f-b069-27f45b8cf1b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.726098 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07663a-5565-4c5f-b069-27f45b8cf1b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:57.726404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07663a-5565-4c5f-b069-27f45b8cf1b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.727810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07663a-5565-4c5f-b069-27f45b8cf1b3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.728217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25f29c-1716-4e7b-a620-cb49adee28b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.729011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25f29c-1716-4e7b-a620-cb49adee28b1 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.729298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25f29c-1716-4e7b-a620-cb49adee28b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.729569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25f29c-1716-4e7b-a620-cb49adee28b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.737089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25f29c-1716-4e7b-a620-cb49adee28b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.738678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25f29c-1716-4e7b-a620-cb49adee28b1 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.740682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 401/808] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:57.741753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07663a-5565-4c5f-b069-27f45b8cf1b3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-9a25f29c-1716-4e7b-a620-cb49adee28b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:57.757586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:57.758692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:57.760002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:57.760002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:57.760002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:57.760002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:57.760002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:57.760002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:57.760002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:57.760002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:57.760002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:41:57.760002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:57.763482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 408/809] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 4 05:41:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:57.764321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25f29c-1716-4e7b-a620-cb49adee28b1 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:57.844822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:57.847908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07663a-5565-4c5f-b069-27f45b8cf1b3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.861218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:57.865026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25f29c-1716-4e7b-a620-cb49adee28b1 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:57.881879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ceeb3-1178-4f5d-8e5a-1f5ddea32026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.882637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ceeb3-1178-4f5d-8e5a-1f5ddea32026 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.882983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ceeb3-1178-4f5d-8e5a-1f5ddea32026 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.883350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ceeb3-1178-4f5d-8e5a-1f5ddea32026 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.884060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ceeb3-1178-4f5d-8e5a-1f5ddea32026 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:57.884414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ceeb3-1178-4f5d-8e5a-1f5ddea32026 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.885791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ceeb3-1178-4f5d-8e5a-1f5ddea32026 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.891582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 402/810] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 4 05:41:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:57.892465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ceeb3-1178-4f5d-8e5a-1f5ddea32026 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:57.900810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ed48a-d577-432b-98eb-798068d512b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:57.901716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ed48a-d577-432b-98eb-798068d512b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.901994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ed48a-d577-432b-98eb-798068d512b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.902267 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ed48a-d577-432b-98eb-798068d512b5 None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/655b3c5b98fa4d91b328d04d42fc263a/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.904362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ed48a-d577-432b-98eb-798068d512b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=655b3c5b98fa4d91b328d04d42fc263a, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:57.905805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ed48a-d577-432b-98eb-798068d512b5 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:57.920453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 409/811] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:41:57 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/655b3c5b98fa4d91b328d04d42fc263a/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:41:57.921691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a38878-84d9-42c0-862d-f018203eb081 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:57.921976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a38878-84d9-42c0-862d-f018203eb081 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:57.922248 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a38878-84d9-42c0-862d-f018203eb081 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:57.923537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 410/812] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:41:57.924247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a38878-84d9-42c0-862d-f018203eb081 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:58.014298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ceeb3-1178-4f5d-8e5a-1f5ddea32026 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:58.051119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe709691-34a1-4e4f-8d2e-28c7345d3ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:58.051984 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe709691-34a1-4e4f-8d2e-28c7345d3ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:58.052264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe709691-34a1-4e4f-8d2e-28c7345d3ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:58.052534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe709691-34a1-4e4f-8d2e-28c7345d3ab9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:58.053097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a38878-84d9-42c0-862d-f018203eb081 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:58.061983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe709691-34a1-4e4f-8d2e-28c7345d3ab9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:58.063488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe709691-34a1-4e4f-8d2e-28c7345d3ab9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-fe709691-34a1-4e4f-8d2e-28c7345d3ab9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:41:58.083055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:58.084175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:58.085148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:58.085148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:58.085148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:41:58.085148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:41:58.085148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:41:58.085148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:41:58.085148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:41:58.085148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:41:58.085148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:41:58.085148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:41:58.085696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 403/813] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 4 05:41:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:58.086275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe709691-34a1-4e4f-8d2e-28c7345d3ab9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:58.095713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6fd0c-c7d8-4d73-9dfc-6eb51d002b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:58.096327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6fd0c-c7d8-4d73-9dfc-6eb51d002b70 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:58.096603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6fd0c-c7d8-4d73-9dfc-6eb51d002b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:58.096872 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6fd0c-c7d8-4d73-9dfc-6eb51d002b70 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:58.097714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6fd0c-c7d8-4d73-9dfc-6eb51d002b70 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:58.098015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6fd0c-c7d8-4d73-9dfc-6eb51d002b70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:58.099491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6fd0c-c7d8-4d73-9dfc-6eb51d002b70 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:58.114638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:58.118770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 411/814] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 4 05:41:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:58.119977 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a76c5-90ff-4eb8-82d6-c3252fe30989 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:58.120262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a76c5-90ff-4eb8-82d6-c3252fe30989 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:58.120531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a76c5-90ff-4eb8-82d6-c3252fe30989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:58.137523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ee9a76c5-90ff-4eb8-82d6-c3252fe30989 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:41:58.138807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ee9a76c5-90ff-4eb8-82d6-c3252fe30989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:41:58.150288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:58.197595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe709691-34a1-4e4f-8d2e-28c7345d3ab9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:58.222413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 412/815] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:41:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:41:58.222413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a76c5-90ff-4eb8-82d6-c3252fe30989 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:58.248748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3bd77-7c60-427f-b074-0141639bcd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:58.249444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3bd77-7c60-427f-b074-0141639bcd37 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:58.249761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3bd77-7c60-427f-b074-0141639bcd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:58.250037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3bd77-7c60-427f-b074-0141639bcd37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:58.250790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3bd77-7c60-427f-b074-0141639bcd37 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:58.252201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3bd77-7c60-427f-b074-0141639bcd37 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:58.275118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 404/816] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 4 05:41:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:41:58.276136 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3bd77-7c60-427f-b074-0141639bcd37 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:58.340203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a76c5-90ff-4eb8-82d6-c3252fe30989 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:58.402985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:58.405239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54735aca-46ba-411d-a675-3b3b4e1b07f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:58.406007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54735aca-46ba-411d-a675-3b3b4e1b07f2 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:58.406352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54735aca-46ba-411d-a675-3b3b4e1b07f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:58.406694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54735aca-46ba-411d-a675-3b3b4e1b07f2 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:58.408227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54735aca-46ba-411d-a675-3b3b4e1b07f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:58.408857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54735aca-46ba-411d-a675-3b3b4e1b07f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:58.411216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54735aca-46ba-411d-a675-3b3b4e1b07f2 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:58.422488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3bd77-7c60-427f-b074-0141639bcd37 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:58.430756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:58.437938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 413/817] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 4 05:41:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:58.443291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54735aca-46ba-411d-a675-3b3b4e1b07f2 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:58.469747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf828fe-eb72-4df9-88ef-92cf86c20fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:58.470824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf828fe-eb72-4df9-88ef-92cf86c20fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:58.471188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf828fe-eb72-4df9-88ef-92cf86c20fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:58.471707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf828fe-eb72-4df9-88ef-92cf86c20fe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:58.553083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf828fe-eb72-4df9-88ef-92cf86c20fe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:58.554762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf828fe-eb72-4df9-88ef-92cf86c20fe3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:58.573408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54735aca-46ba-411d-a675-3b3b4e1b07f2 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:58.614165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f8f83-1af5-4253-bcb2-349c02d64215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:58.615156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f8f83-1af5-4253-bcb2-349c02d64215 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:58.615535 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f8f83-1af5-4253-bcb2-349c02d64215 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:58.615900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f8f83-1af5-4253-bcb2-349c02d64215 None admin] PATH_INFO: `/v3/services/network` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:58.616668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f8f83-1af5-4253-bcb2-349c02d64215 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:58.618134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f8f83-1af5-4253-bcb2-349c02d64215 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-773f8f83-1af5-4253-bcb2-349c02d64215 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:58.630543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:58.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 05:41:58.632781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:58.634957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 414/818] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:58 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:58.640222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f8f83-1af5-4253-bcb2-349c02d64215 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:58.681818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:58.683394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 405/819] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:41:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:41:58.684501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf828fe-eb72-4df9-88ef-92cf86c20fe3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:58.712117 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:58.752777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f8f83-1af5-4253-bcb2-349c02d64215 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:58.787766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a78b8-011c-4d0b-925a-c25fe6ea500f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:58.788755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a78b8-011c-4d0b-925a-c25fe6ea500f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:58.788867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a78b8-011c-4d0b-925a-c25fe6ea500f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:58.789131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a78b8-011c-4d0b-925a-c25fe6ea500f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:58.800718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a78b8-011c-4d0b-925a-c25fe6ea500f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:58.802482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a78b8-011c-4d0b-925a-c25fe6ea500f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:58.818836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf828fe-eb72-4df9-88ef-92cf86c20fe3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-bd0a78b8-011c-4d0b-925a-c25fe6ea500f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:58.823137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:58.824092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:58.824980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:58.824980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:58.824980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:58.824980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:58.829100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 415/820] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:58.829954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a78b8-011c-4d0b-925a-c25fe6ea500f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:58.864915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d387a3-beb7-4c06-be23-d1fa50ce8fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:58.866387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d387a3-beb7-4c06-be23-d1fa50ce8fcf None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:58.866816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d387a3-beb7-4c06-be23-d1fa50ce8fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:58.867485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d387a3-beb7-4c06-be23-d1fa50ce8fcf None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:58.868361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d387a3-beb7-4c06-be23-d1fa50ce8fcf None admin] RBAC: Adding query filter params (name=network) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:58.868874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d387a3-beb7-4c06-be23-d1fa50ce8fcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:58.871549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d387a3-beb7-4c06-be23-d1fa50ce8fcf None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:58.877269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 406/821] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:41:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:58.878544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d387a3-beb7-4c06-be23-d1fa50ce8fcf None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:58.930885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:58.960852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a78b8-011c-4d0b-925a-c25fe6ea500f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:58.983768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:58.988806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d387a3-beb7-4c06-be23-d1fa50ce8fcf None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:59.014935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12836b57-a220-4dc0-9182-07f1c58ad805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:59.016182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12836b57-a220-4dc0-9182-07f1c58ad805 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:59.016468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12836b57-a220-4dc0-9182-07f1c58ad805 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:59.016745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12836b57-a220-4dc0-9182-07f1c58ad805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:59.026670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f0085-b05a-460b-99d9-f232f14b6da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:59.027717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f0085-b05a-460b-99d9-f232f14b6da0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:59.028214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f0085-b05a-460b-99d9-f232f14b6da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:59.028672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f0085-b05a-460b-99d9-f232f14b6da0 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:59.029583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f0085-b05a-460b-99d9-f232f14b6da0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:59.030073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f0085-b05a-460b-99d9-f232f14b6da0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:59.031291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12836b57-a220-4dc0-9182-07f1c58ad805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:59.031920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f0085-b05a-460b-99d9-f232f14b6da0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:59.032769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12836b57-a220-4dc0-9182-07f1c58ad805 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:59.052258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 407/822] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:59.053468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f0085-b05a-460b-99d9-f232f14b6da0 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-12836b57-a220-4dc0-9182-07f1c58ad805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:41:59.064682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:59.065956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:59.066842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:41:59.066842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:41:59.066842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:41:59.066842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:59.070449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 416/823] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:59.071390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12836b57-a220-4dc0-9182-07f1c58ad805 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:59.183702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12836b57-a220-4dc0-9182-07f1c58ad805 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:59.186967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f0085-b05a-460b-99d9-f232f14b6da0 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:59.189330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:59.221345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79cf0b-221c-45ae-a354-52e6713543c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:59.222077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79cf0b-221c-45ae-a354-52e6713543c1 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:59.222352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79cf0b-221c-45ae-a354-52e6713543c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:59.222623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79cf0b-221c-45ae-a354-52e6713543c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:59.229620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79cf0b-221c-45ae-a354-52e6713543c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:59.230001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166e8cb-9072-464f-84bc-f0e56fe2dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:59.230673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166e8cb-9072-464f-84bc-f0e56fe2dce1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:59.230951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166e8cb-9072-464f-84bc-f0e56fe2dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:59.231110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79cf0b-221c-45ae-a354-52e6713543c1 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:59.231291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166e8cb-9072-464f-84bc-f0e56fe2dce1 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:59.232189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166e8cb-9072-464f-84bc-f0e56fe2dce1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:59.232491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166e8cb-9072-464f-84bc-f0e56fe2dce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:59.234471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166e8cb-9072-464f-84bc-f0e56fe2dce1 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:59.240048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 408/824] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:41:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:59.240826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166e8cb-9072-464f-84bc-f0e56fe2dce1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-ce79cf0b-221c-45ae-a354-52e6713543c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:59.251383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:59.253360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:59.255459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:59.255459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:59.255459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:59.255459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:59.255459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:59.255459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:59.255459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:59.255459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:59.263283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 417/825] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:59.263283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79cf0b-221c-45ae-a354-52e6713543c1 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:59.287670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:59.361740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79cf0b-221c-45ae-a354-52e6713543c1 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:59.366094 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166e8cb-9072-464f-84bc-f0e56fe2dce1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:59.396647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b351c-ad64-48c1-b39c-7fd3e23928cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:59.397492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b351c-ad64-48c1-b39c-7fd3e23928cc None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:59.397867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b351c-ad64-48c1-b39c-7fd3e23928cc None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:59.398235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b351c-ad64-48c1-b39c-7fd3e23928cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:59.399130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b351c-ad64-48c1-b39c-7fd3e23928cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:59.400840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b351c-ad64-48c1-b39c-7fd3e23928cc None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:59.406343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6df2d-5770-4107-84fc-9d93c29686bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:59.407274 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6df2d-5770-4107-84fc-9d93c29686bd None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:59.407581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6df2d-5770-4107-84fc-9d93c29686bd None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:59.407859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6df2d-5770-4107-84fc-9d93c29686bd None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:59.408583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6df2d-5770-4107-84fc-9d93c29686bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:59.408884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6df2d-5770-4107-84fc-9d93c29686bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:59.410419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6df2d-5770-4107-84fc-9d93c29686bd None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:59.428369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 409/826] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:59.429254 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6df2d-5770-4107-84fc-9d93c29686bd None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:59.435567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 418/827] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 4 05:41:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:41:59.437027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b351c-ad64-48c1-b39c-7fd3e23928cc None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:59.462740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:59.545618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:59.571345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b351c-ad64-48c1-b39c-7fd3e23928cc None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:59.611027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5a166-5f98-49c5-a3a7-01e0d604ef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:59.611909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5a166-5f98-49c5-a3a7-01e0d604ef6d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:59.612188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5a166-5f98-49c5-a3a7-01e0d604ef6d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:59.612464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5a166-5f98-49c5-a3a7-01e0d604ef6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:59.615411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6df2d-5770-4107-84fc-9d93c29686bd None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:59.630205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5a166-5f98-49c5-a3a7-01e0d604ef6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:59.631760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5a166-5f98-49c5-a3a7-01e0d604ef6d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-0ef5a166-5f98-49c5-a3a7-01e0d604ef6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:41:59.654215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:41:59.655194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:41:59.656113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:41:59.656113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:41:59.656113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:41:59.656113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:41:59.656113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:41:59.656113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:41:59.656113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:41:59.656113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:41:59.664356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 419/828] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:41:59.668624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5a166-5f98-49c5-a3a7-01e0d604ef6d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:59.689780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8721526-b17c-4ab6-aaaf-d85250a70309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:59.690462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8721526-b17c-4ab6-aaaf-d85250a70309 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:59.690740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8721526-b17c-4ab6-aaaf-d85250a70309 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:59.691009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8721526-b17c-4ab6-aaaf-d85250a70309 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:59.691840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8721526-b17c-4ab6-aaaf-d85250a70309 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:59.692146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8721526-b17c-4ab6-aaaf-d85250a70309 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:59.693893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8721526-b17c-4ab6-aaaf-d85250a70309 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:59.713882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 410/829] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:41:59.720842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8721526-b17c-4ab6-aaaf-d85250a70309 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:59.800160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5a166-5f98-49c5-a3a7-01e0d604ef6d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:59.841400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:59.849188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba3cc8-7987-4631-91c7-89503cab5ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:59.849877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba3cc8-7987-4631-91c7-89503cab5ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:59.850162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba3cc8-7987-4631-91c7-89503cab5ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:59.850432 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba3cc8-7987-4631-91c7-89503cab5ca4 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:59.851235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba3cc8-7987-4631-91c7-89503cab5ca4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:41:59.851579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba3cc8-7987-4631-91c7-89503cab5ca4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:59.853093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba3cc8-7987-4631-91c7-89503cab5ca4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:59.865088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:41:59.867550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8721526-b17c-4ab6-aaaf-d85250a70309 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:41:59.869862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 420/830] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:41:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:41:59.876236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba3cc8-7987-4631-91c7-89503cab5ca4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:41:59.911896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a4895-855a-4316-84a7-e63a983945ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:41:59.913172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a4895-855a-4316-84a7-e63a983945ad None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:41:59.913687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a4895-855a-4316-84a7-e63a983945ad None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:41:59.914204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a4895-855a-4316-84a7-e63a983945ad None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/884ed6ddb71647088b33dc9623143b9c/roles/04f8b72053764c7f8067d4b478d31e95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:41:59.931122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a4895-855a-4316-84a7-e63a983945ad None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=884ed6ddb71647088b33dc9623143b9c, role_id=04f8b72053764c7f8067d4b478d31e95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:41:59.933070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a4895-855a-4316-84a7-e63a983945ad None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:41:59.953257 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 411/831] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:41:59 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/884ed6ddb71647088b33dc9623143b9c/roles/04f8b72053764c7f8067d4b478d31e95 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:41:59.995181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba3cc8-7987-4631-91c7-89503cab5ca4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:00.022151 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f31623-3e57-4233-b8ed-bf4790fe61b8 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.022506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f31623-3e57-4233-b8ed-bf4790fe61b8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.022694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f31623-3e57-4233-b8ed-bf4790fe61b8 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.024382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 412/832] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:00.030004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c704a-6b27-4261-857b-1a60b241acba None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.030414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c704a-6b27-4261-857b-1a60b241acba None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.030594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c704a-6b27-4261-857b-1a60b241acba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.033879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdd390-c9d5-4945-abce-3b0b23233c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:00.034558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdd390-c9d5-4945-abce-3b0b23233c76 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.034839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdd390-c9d5-4945-abce-3b0b23233c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.035113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdd390-c9d5-4945-abce-3b0b23233c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.036122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdd390-c9d5-4945-abce-3b0b23233c76 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=2cb245e4300e4d5fbdd425e936f4ac77) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:00.036436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdd390-c9d5-4945-abce-3b0b23233c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:00.038017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdd390-c9d5-4945-abce-3b0b23233c76 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:00.047260 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 421/833] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:41:59 2022] GET /identity/v3/role_assignments?user.id=2cb245e4300e4d5fbdd425e936f4ac77&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:00.051495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-8e8c704a-6b27-4261-857b-1a60b241acba None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:00.052662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8e8c704a-6b27-4261-857b-1a60b241acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:00.133501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 413/834] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:00.157647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdd390-c9d5-4945-abce-3b0b23233c76 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:00.228056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:00.268436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdd390-c9d5-4945-abce-3b0b23233c76 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:00.308278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d853c-d913-4d3b-96c5-a2b54083cce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:00.308996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d853c-d913-4d3b-96c5-a2b54083cce5 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.309272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d853c-d913-4d3b-96c5-a2b54083cce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.309566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d853c-d913-4d3b-96c5-a2b54083cce5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.320840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d853c-d913-4d3b-96c5-a2b54083cce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-7b1d853c-d913-4d3b-96c5-a2b54083cce5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:00.328742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:00.329714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:00.330566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 422/835] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:42:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:00.336126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c704a-6b27-4261-857b-1a60b241acba None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:00.411774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:00.446321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c704a-6b27-4261-857b-1a60b241acba None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:00.476185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ac7b6-b72b-4c17-8b8c-f8091eff7900 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.476513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ac7b6-b72b-4c17-8b8c-f8091eff7900 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.476738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ac7b6-b72b-4c17-8b8c-f8091eff7900 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.478570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 423/836] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:00.481035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b3c89-ec3b-4691-8747-2ae717491d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:00.481721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b3c89-ec3b-4691-8747-2ae717491d17 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.482077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b3c89-ec3b-4691-8747-2ae717491d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.482358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b3c89-ec3b-4691-8747-2ae717491d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.483116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b3c89-ec3b-4691-8747-2ae717491d17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:00.483447 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b3c89-ec3b-4691-8747-2ae717491d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:00.483752 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40701906-b1dd-42e3-9009-bafa9bbed91c None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.484124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40701906-b1dd-42e3-9009-bafa9bbed91c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.484488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40701906-b1dd-42e3-9009-bafa9bbed91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.484832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b3c89-ec3b-4691-8747-2ae717491d17 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:00.498449 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 414/837] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:42:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:00.499746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b3c89-ec3b-4691-8747-2ae717491d17 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:00.503325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-40701906-b1dd-42e3-9009-bafa9bbed91c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:00.504625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-40701906-b1dd-42e3-9009-bafa9bbed91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:00.591427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 424/838] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:00.613895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40701906-b1dd-42e3-9009-bafa9bbed91c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:00.623516 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b3c89-ec3b-4691-8747-2ae717491d17 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:00.658125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:00.666453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab86ac-02e5-4a95-8fe8-07c90fe06a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:00.667507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab86ac-02e5-4a95-8fe8-07c90fe06a74 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.667924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab86ac-02e5-4a95-8fe8-07c90fe06a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.668794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab86ac-02e5-4a95-8fe8-07c90fe06a74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.682729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab86ac-02e5-4a95-8fe8-07c90fe06a74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:00.683605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:00.684738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab86ac-02e5-4a95-8fe8-07c90fe06a74 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-15ab86ac-02e5-4a95-8fe8-07c90fe06a74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:00.707917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:00.708882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:00.709789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:00.709789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:00.709789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:00.709789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:00.712442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 415/839] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:00.720567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab86ac-02e5-4a95-8fe8-07c90fe06a74 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:00.726606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40701906-b1dd-42e3-9009-bafa9bbed91c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:00.764185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6544c4-88e3-45d0-b8b8-6cd6268a4d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:00.765309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6544c4-88e3-45d0-b8b8-6cd6268a4d20 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.765723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6544c4-88e3-45d0-b8b8-6cd6268a4d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.766127 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6544c4-88e3-45d0-b8b8-6cd6268a4d20 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.766910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6544c4-88e3-45d0-b8b8-6cd6268a4d20 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:00.768708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6544c4-88e3-45d0-b8b8-6cd6268a4d20 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-2e6544c4-88e3-45d0-b8b8-6cd6268a4d20 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:00.781787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:00.782897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 05:42:00.783751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:00.791353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 425/840] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:42:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:00.792397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6544c4-88e3-45d0-b8b8-6cd6268a4d20 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:00.826678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab86ac-02e5-4a95-8fe8-07c90fe06a74 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:00.861013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa60fce-bbc1-4320-aee6-6e1f071d8ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:00.861907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa60fce-bbc1-4320-aee6-6e1f071d8ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.862289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa60fce-bbc1-4320-aee6-6e1f071d8ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.862757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa60fce-bbc1-4320-aee6-6e1f071d8ca9 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.863677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa60fce-bbc1-4320-aee6-6e1f071d8ca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:00.864134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa60fce-bbc1-4320-aee6-6e1f071d8ca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:00.865625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa60fce-bbc1-4320-aee6-6e1f071d8ca9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:00.882627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 416/841] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:00.884079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa60fce-bbc1-4320-aee6-6e1f071d8ca9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:00.902027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6544c4-88e3-45d0-b8b8-6cd6268a4d20 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:00.920920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:00.928884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:00.942554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900663-9adf-4665-932e-d51e468e4816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:00.943521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900663-9adf-4665-932e-d51e468e4816 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:00.944316 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900663-9adf-4665-932e-d51e468e4816 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:00.944757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900663-9adf-4665-932e-d51e468e4816 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:00.945730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22900663-9adf-4665-932e-d51e468e4816 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:00.946318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22900663-9adf-4665-932e-d51e468e4816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:00.948118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22900663-9adf-4665-932e-d51e468e4816 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:00.958247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 426/842] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:42:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:00.959420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900663-9adf-4665-932e-d51e468e4816 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:00.989213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa60fce-bbc1-4320-aee6-6e1f071d8ca9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:01.023695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423b6e6-7f1d-48a8-b7db-6a6e730dd0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:01.024743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423b6e6-7f1d-48a8-b7db-6a6e730dd0ad None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:01.025152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423b6e6-7f1d-48a8-b7db-6a6e730dd0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:01.025487 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423b6e6-7f1d-48a8-b7db-6a6e730dd0ad None admin] PATH_INFO: `/v3/users/swift` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:01.033119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423b6e6-7f1d-48a8-b7db-6a6e730dd0ad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:01.035015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423b6e6-7f1d-48a8-b7db-6a6e730dd0ad None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-6423b6e6-7f1d-48a8-b7db-6a6e730dd0ad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:42:01.050248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:01.051196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:01.052180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:01.052180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:01.052180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:42:01.052180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:42:01.052180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:42:01.052180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:42:01.052180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:42:01.052180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:42:01.052180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 05:42:01.052180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:01.056515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 417/843] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:42:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:01.057731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423b6e6-7f1d-48a8-b7db-6a6e730dd0ad None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:01.069381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900663-9adf-4665-932e-d51e468e4816 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:01.106094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ed622-9f7d-4ee7-89b4-55e81e6466a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:01.107218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ed622-9f7d-4ee7-89b4-55e81e6466a1 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:01.107719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ed622-9f7d-4ee7-89b4-55e81e6466a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:01.108230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ed622-9f7d-4ee7-89b4-55e81e6466a1 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:01.109150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ed622-9f7d-4ee7-89b4-55e81e6466a1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:01.109678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ed622-9f7d-4ee7-89b4-55e81e6466a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:01.111529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ed622-9f7d-4ee7-89b4-55e81e6466a1 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:01.123571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 427/844] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:42:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:01.168782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423b6e6-7f1d-48a8-b7db-6a6e730dd0ad None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:01.170472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:01.208688 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167548a-3791-4091-8bd2-8ee75c48d14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:01.209476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167548a-3791-4091-8bd2-8ee75c48d14c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:01.209761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167548a-3791-4091-8bd2-8ee75c48d14c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:01.210033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167548a-3791-4091-8bd2-8ee75c48d14c None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:01.210985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167548a-3791-4091-8bd2-8ee75c48d14c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:01.211324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167548a-3791-4091-8bd2-8ee75c48d14c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:01.212903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167548a-3791-4091-8bd2-8ee75c48d14c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:01.237951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 418/845] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 4 05:42:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:01.240430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ed622-9f7d-4ee7-89b4-55e81e6466a1 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:01.293442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:01.356137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ed622-9f7d-4ee7-89b4-55e81e6466a1 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:01.389937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fae787-f802-4846-a04f-a27efa5ba1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:01.390519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fae787-f802-4846-a04f-a27efa5ba1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:01.390844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fae787-f802-4846-a04f-a27efa5ba1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:01.391118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fae787-f802-4846-a04f-a27efa5ba1e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:01.458167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fae787-f802-4846-a04f-a27efa5ba1e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:01.459668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fae787-f802-4846-a04f-a27efa5ba1e2 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:01.511627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:01.571733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 428/846] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:42:01.577255 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167548a-3791-4091-8bd2-8ee75c48d14c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:01.687595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167548a-3791-4091-8bd2-8ee75c48d14c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:01.723647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67b2f1-1d4c-4b54-bcba-482f8979f744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:01.724456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67b2f1-1d4c-4b54-bcba-482f8979f744 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:01.724738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67b2f1-1d4c-4b54-bcba-482f8979f744 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:01.725009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67b2f1-1d4c-4b54-bcba-482f8979f744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:01.735918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67b2f1-1d4c-4b54-bcba-482f8979f744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:01.737386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67b2f1-1d4c-4b54-bcba-482f8979f744 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-cc67b2f1-1d4c-4b54-bcba-482f8979f744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:01.755285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:01.756180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:01.757076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:01.757076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:01.757076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:01.757076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:01.757076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 419/847] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:01.762401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fae787-f802-4846-a04f-a27efa5ba1e2 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:01.868535 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fae787-f802-4846-a04f-a27efa5ba1e2 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:01.902626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dea47-c773-431a-8023-e15d656f75da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:01.903315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dea47-c773-431a-8023-e15d656f75da None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:01.903596 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dea47-c773-431a-8023-e15d656f75da None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:01.903866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dea47-c773-431a-8023-e15d656f75da None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:01.904588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dea47-c773-431a-8023-e15d656f75da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:01.904886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dea47-c773-431a-8023-e15d656f75da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:01.906275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dea47-c773-431a-8023-e15d656f75da None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:01.918650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 429/848] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:01.924242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67b2f1-1d4c-4b54-bcba-482f8979f744 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:01.941101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:02.031683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67b2f1-1d4c-4b54-bcba-482f8979f744 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:02.065890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64875a1-5445-42bd-9961-7a35da2ecf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:02.066755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64875a1-5445-42bd-9961-7a35da2ecf06 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.067042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64875a1-5445-42bd-9961-7a35da2ecf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.067351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64875a1-5445-42bd-9961-7a35da2ecf06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.074483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64875a1-5445-42bd-9961-7a35da2ecf06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:02.075968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64875a1-5445-42bd-9961-7a35da2ecf06 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:02.088505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6861798-11d9-4410-8076-da42fd89b330 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.088770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6861798-11d9-4410-8076-da42fd89b330 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.089039 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6861798-11d9-4410-8076-da42fd89b330 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-c64875a1-5445-42bd-9961-7a35da2ecf06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:42:02.093631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:02.094535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:02.096579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:42:02.096579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:42:02.096579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:42:02.096579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:42:02.096579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:42:02.096579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:42:02.096579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:02.096579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:02.096579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 420/850] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:02.097067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 430/850] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:42:02.097137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241e89c-41f4-491f-b291-94f9e4dac41e None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.097137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241e89c-41f4-491f-b291-94f9e4dac41e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.097137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241e89c-41f4-491f-b291-94f9e4dac41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.102291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6861798-11d9-4410-8076-da42fd89b330 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:02.112298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:02.117338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-9241e89c-41f4-491f-b291-94f9e4dac41e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:02.118824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-9241e89c-41f4-491f-b291-94f9e4dac41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:02.199674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 421/851] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:02.222635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241e89c-41f4-491f-b291-94f9e4dac41e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:02.225051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:02.232041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6861798-11d9-4410-8076-da42fd89b330 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:02.270329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaaece-b030-4d69-b4d3-8d70564c505c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:02.272011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaaece-b030-4d69-b4d3-8d70564c505c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.272290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaaece-b030-4d69-b4d3-8d70564c505c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.272558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaaece-b030-4d69-b4d3-8d70564c505c None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.273355 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaaece-b030-4d69-b4d3-8d70564c505c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:02.273658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaaece-b030-4d69-b4d3-8d70564c505c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:02.275161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaaece-b030-4d69-b4d3-8d70564c505c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:02.293590 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 431/852] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:02.297057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc4fd5-cd71-4cb5-a230-2a7a321c8f65 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.297703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc4fd5-cd71-4cb5-a230-2a7a321c8f65 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.298524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc4fd5-cd71-4cb5-a230-2a7a321c8f65 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.301625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 432/853] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:02.304828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc4fd5-cd71-4cb5-a230-2a7a321c8f65 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:02.350945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241e89c-41f4-491f-b291-94f9e4dac41e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:02.387920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea60325-0642-4d39-8c8f-df178f244e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:02.388598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:02.388838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea60325-0642-4d39-8c8f-df178f244e55 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.389119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea60325-0642-4d39-8c8f-df178f244e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.389388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea60325-0642-4d39-8c8f-df178f244e55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.401473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea60325-0642-4d39-8c8f-df178f244e55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-0ea60325-0642-4d39-8c8f-df178f244e55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:02.409925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:02.410848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:02.411706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 422/854] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:42:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:02.413115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eb3c1-b0bd-4402-99d1-c13733075650 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.413115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eb3c1-b0bd-4402-99d1-c13733075650 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.413271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eb3c1-b0bd-4402-99d1-c13733075650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.435058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-b29eb3c1-b0bd-4402-99d1-c13733075650 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:02.436282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b29eb3c1-b0bd-4402-99d1-c13733075650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:02.437455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc4fd5-cd71-4cb5-a230-2a7a321c8f65 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:02.483255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cf6fd-9be7-4988-b8a8-6d7e780e3a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:02.483942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cf6fd-9be7-4988-b8a8-6d7e780e3a20 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.484218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cf6fd-9be7-4988-b8a8-6d7e780e3a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.484486 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cf6fd-9be7-4988-b8a8-6d7e780e3a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.485319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cf6fd-9be7-4988-b8a8-6d7e780e3a20 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=655b3c5b98fa4d91b328d04d42fc263a) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:02.485625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cf6fd-9be7-4988-b8a8-6d7e780e3a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:02.487060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cf6fd-9be7-4988-b8a8-6d7e780e3a20 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:02.499283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 433/855] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:02 2022] GET /identity/v3/role_assignments?user.id=655b3c5b98fa4d91b328d04d42fc263a&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:02.499283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cf6fd-9be7-4988-b8a8-6d7e780e3a20 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:02.500666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:02.531276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 423/856] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:02.543902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29eb3c1-b0bd-4402-99d1-c13733075650 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:02.622662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cf6fd-9be7-4988-b8a8-6d7e780e3a20 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:02.663008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:02.672086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2373d-0f94-4714-b7e0-878cec7e1b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:02.672689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29eb3c1-b0bd-4402-99d1-c13733075650 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:02.673140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2373d-0f94-4714-b7e0-878cec7e1b4c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.673140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2373d-0f94-4714-b7e0-878cec7e1b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.673290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2373d-0f94-4714-b7e0-878cec7e1b4c None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.674020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2373d-0f94-4714-b7e0-878cec7e1b4c None admin] RBAC: Adding query filter params (name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:02.674322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2373d-0f94-4714-b7e0-878cec7e1b4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:02.675937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2373d-0f94-4714-b7e0-878cec7e1b4c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:02.690861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 434/857] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:42:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:02.692120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2373d-0f94-4714-b7e0-878cec7e1b4c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:02.720048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfe48f-2cbf-4f03-884f-262bfec29f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:02.721026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfe48f-2cbf-4f03-884f-262bfec29f8a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.721138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfe48f-2cbf-4f03-884f-262bfec29f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.721410 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfe48f-2cbf-4f03-884f-262bfec29f8a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.722064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfe48f-2cbf-4f03-884f-262bfec29f8a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:02.724127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfe48f-2cbf-4f03-884f-262bfec29f8a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-c0dfe48f-2cbf-4f03-884f-262bfec29f8a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:02.739388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:02.740357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:02.741196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 424/858] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Jul 4 05:42:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:02.746877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfe48f-2cbf-4f03-884f-262bfec29f8a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:02.800222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2373d-0f94-4714-b7e0-878cec7e1b4c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:02.808073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:02.835863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437da91-f7f1-4a5e-9f73-a584f92aae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:02.836768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437da91-f7f1-4a5e-9f73-a584f92aae92 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.837043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437da91-f7f1-4a5e-9f73-a584f92aae92 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.837359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437da91-f7f1-4a5e-9f73-a584f92aae92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.848349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfe48f-2cbf-4f03-884f-262bfec29f8a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:02.849941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437da91-f7f1-4a5e-9f73-a584f92aae92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:02.851547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437da91-f7f1-4a5e-9f73-a584f92aae92 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-6437da91-f7f1-4a5e-9f73-a584f92aae92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:02.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:02.873283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:02.874229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:02.874229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:02.874229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:02.874229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:02.874229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 435/859] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:02.880814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437da91-f7f1-4a5e-9f73-a584f92aae92 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:02.888374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484c4cd-9ace-4763-8bbd-9884aef744d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:02.889176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484c4cd-9ace-4763-8bbd-9884aef744d7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:02.889550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484c4cd-9ace-4763-8bbd-9884aef744d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:02.889917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484c4cd-9ace-4763-8bbd-9884aef744d7 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:02.890745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484c4cd-9ace-4763-8bbd-9884aef744d7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:02.891149 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484c4cd-9ace-4763-8bbd-9884aef744d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:02.892752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484c4cd-9ace-4763-8bbd-9884aef744d7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:02.898548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 425/860] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:02.905102 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484c4cd-9ace-4763-8bbd-9884aef744d7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:03.019314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484c4cd-9ace-4763-8bbd-9884aef744d7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:03.039506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:03.055063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8d79d-d05f-4222-b4f5-4a4c8de45ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:03.055861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8d79d-d05f-4222-b4f5-4a4c8de45ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:03.056345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8d79d-d05f-4222-b4f5-4a4c8de45ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:03.056805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8d79d-d05f-4222-b4f5-4a4c8de45ff7 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:03.057604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8d79d-d05f-4222-b4f5-4a4c8de45ff7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:03.057971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8d79d-d05f-4222-b4f5-4a4c8de45ff7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:03.059928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8d79d-d05f-4222-b4f5-4a4c8de45ff7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:03.070136 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 426/861] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:03.070741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:03.128562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437da91-f7f1-4a5e-9f73-a584f92aae92 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:03.166929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ce58e-dbb4-4d26-88f4-b8dd97c5c6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:03.167652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ce58e-dbb4-4d26-88f4-b8dd97c5c6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:03.167927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ce58e-dbb4-4d26-88f4-b8dd97c5c6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:03.168196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ce58e-dbb4-4d26-88f4-b8dd97c5c6a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:03.168935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ce58e-dbb4-4d26-88f4-b8dd97c5c6a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:03.169234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ce58e-dbb4-4d26-88f4-b8dd97c5c6a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:03.170651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ce58e-dbb4-4d26-88f4-b8dd97c5c6a0 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:03.183995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 436/862] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:03.189443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8d79d-d05f-4222-b4f5-4a4c8de45ff7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:03.242166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccc3d2-a9ad-4c69-a223-d2b1c24b019e None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:03.242457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccc3d2-a9ad-4c69-a223-d2b1c24b019e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:03.242728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccc3d2-a9ad-4c69-a223-d2b1c24b019e None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:03.244411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 437/863] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:42:03.251114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573758db-30a9-471d-a3da-7d2ec591bebf None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:03.252063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573758db-30a9-471d-a3da-7d2ec591bebf None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:03.252350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573758db-30a9-471d-a3da-7d2ec591bebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:03.277252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-573758db-30a9-471d-a3da-7d2ec591bebf None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:03.279397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-573758db-30a9-471d-a3da-7d2ec591bebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:03.300455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8d79d-d05f-4222-b4f5-4a4c8de45ff7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:03.335472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ed9cb-b8f0-47ac-bcf7-7d4cb042f52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:03.336642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ed9cb-b8f0-47ac-bcf7-7d4cb042f52d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:03.337097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ed9cb-b8f0-47ac-bcf7-7d4cb042f52d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:03.337675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ed9cb-b8f0-47ac-bcf7-7d4cb042f52d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:03.346006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ed9cb-b8f0-47ac-bcf7-7d4cb042f52d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:03.347739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ed9cb-b8f0-47ac-bcf7-7d4cb042f52d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:03.355519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 438/864] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-439ed9cb-b8f0-47ac-bcf7-7d4cb042f52d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:42:03.363084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:03.364030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:03.365001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:03.365001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:03.365001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:42:03.365001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:42:03.365001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:42:03.365001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:42:03.365001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:42:03.365001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:42:03.365001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:42:03.365001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:03.367634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 427/865] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 4 05:42:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:03.373584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573758db-30a9-471d-a3da-7d2ec591bebf None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:03.377179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ed9cb-b8f0-47ac-bcf7-7d4cb042f52d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:03.411722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:03.437912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:03.485547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573758db-30a9-471d-a3da-7d2ec591bebf None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:03.511161 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ed9cb-b8f0-47ac-bcf7-7d4cb042f52d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:03.531696 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e558d-a9c8-4ffa-8fcb-0074e2cc4d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:03.532587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e558d-a9c8-4ffa-8fcb-0074e2cc4d1e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:03.533003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e558d-a9c8-4ffa-8fcb-0074e2cc4d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:03.533431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e558d-a9c8-4ffa-8fcb-0074e2cc4d1e None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:03.534469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e558d-a9c8-4ffa-8fcb-0074e2cc4d1e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:03.534966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e558d-a9c8-4ffa-8fcb-0074e2cc4d1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:03.536645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e558d-a9c8-4ffa-8fcb-0074e2cc4d1e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:03.553966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8fcf2-1ae7-41a6-8749-d474f8ca0dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:03.554643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8fcf2-1ae7-41a6-8749-d474f8ca0dcd None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:03.554920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8fcf2-1ae7-41a6-8749-d474f8ca0dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:03.555187 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8fcf2-1ae7-41a6-8749-d474f8ca0dcd None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:03.555958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8fcf2-1ae7-41a6-8749-d474f8ca0dcd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:03.556840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 439/866] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 4 05:42:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:03.557523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8fcf2-1ae7-41a6-8749-d474f8ca0dcd None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:03.564329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e558d-a9c8-4ffa-8fcb-0074e2cc4d1e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:03.590720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 428/867] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:42:03 2022] POST /identity/v3/services => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:42:03.689407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:03.700017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e558d-a9c8-4ffa-8fcb-0074e2cc4d1e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:03.749630 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8fb17-da04-4c3b-a530-0d43e9b544aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:03.750292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8fb17-da04-4c3b-a530-0d43e9b544aa None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:03.750568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8fb17-da04-4c3b-a530-0d43e9b544aa None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:03.750834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8fb17-da04-4c3b-a530-0d43e9b544aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:03.824287 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8fb17-da04-4c3b-a530-0d43e9b544aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:03.826107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8fb17-da04-4c3b-a530-0d43e9b544aa None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:03.915275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:03.930180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 440/868] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:03 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 05:42:03.936184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8fcf2-1ae7-41a6-8749-d474f8ca0dcd None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:04.039143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8fcf2-1ae7-41a6-8749-d474f8ca0dcd None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:04.042006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:04.075060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc31b2-4cb3-4d2f-9f8d-a5d5b77c3fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:04.076271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc31b2-4cb3-4d2f-9f8d-a5d5b77c3fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:04.076747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc31b2-4cb3-4d2f-9f8d-a5d5b77c3fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:04.077229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc31b2-4cb3-4d2f-9f8d-a5d5b77c3fe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:04.088369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc31b2-4cb3-4d2f-9f8d-a5d5b77c3fe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:04.089850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc31b2-4cb3-4d2f-9f8d-a5d5b77c3fe7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-eedc31b2-4cb3-4d2f-9f8d-a5d5b77c3fe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:04.111822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:04.112733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:04.113626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:04.113626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:04.113626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:04.113626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:04.113626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 429/869] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:04.120168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8fb17-da04-4c3b-a530-0d43e9b544aa None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:04.222122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8fb17-da04-4c3b-a530-0d43e9b544aa None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:04.258659 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc52e2-f77f-48da-a5ab-f04e30afa1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:04.259951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc52e2-f77f-48da-a5ab-f04e30afa1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:04.260428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc52e2-f77f-48da-a5ab-f04e30afa1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:04.260887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc52e2-f77f-48da-a5ab-f04e30afa1c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:04.261848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc52e2-f77f-48da-a5ab-f04e30afa1c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:04.262381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc52e2-f77f-48da-a5ab-f04e30afa1c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:04.263978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc52e2-f77f-48da-a5ab-f04e30afa1c0 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:04.277062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 441/870] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:04.282826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc31b2-4cb3-4d2f-9f8d-a5d5b77c3fe7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:04.388343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc31b2-4cb3-4d2f-9f8d-a5d5b77c3fe7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:04.422283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b8caa-8aee-4354-9675-d5dc381b09ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:04.423088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b8caa-8aee-4354-9675-d5dc381b09ef None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:04.423400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b8caa-8aee-4354-9675-d5dc381b09ef None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:04.423677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b8caa-8aee-4354-9675-d5dc381b09ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:04.430478 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b8caa-8aee-4354-9675-d5dc381b09ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:04.431993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b8caa-8aee-4354-9675-d5dc381b09ef None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-387b8caa-8aee-4354-9675-d5dc381b09ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:42:04.450885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:04.451842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:04.452712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:42:04.452712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:42:04.452712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:42:04.452712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:42:04.452712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:42:04.452712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:42:04.452712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:04.452712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:04.454913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 430/871] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:04.459944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc52e2-f77f-48da-a5ab-f04e30afa1c0 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:04.513679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:04.568474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc52e2-f77f-48da-a5ab-f04e30afa1c0 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:04.603575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99046e01-4dd6-43c2-826a-5ebc06b59a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:04.604583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99046e01-4dd6-43c2-826a-5ebc06b59a66 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:04.605054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99046e01-4dd6-43c2-826a-5ebc06b59a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:04.605541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99046e01-4dd6-43c2-826a-5ebc06b59a66 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:04.606591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99046e01-4dd6-43c2-826a-5ebc06b59a66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:04.607103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99046e01-4dd6-43c2-826a-5ebc06b59a66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:04.608796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99046e01-4dd6-43c2-826a-5ebc06b59a66 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:04.623965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 442/872] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:04.630461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b8caa-8aee-4354-9675-d5dc381b09ef None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:04.650903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:04.689548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f4d80-9a80-458e-9a29-a9d4c63b6fb2 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:04.689877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f4d80-9a80-458e-9a29-a9d4c63b6fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:04.690089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f4d80-9a80-458e-9a29-a9d4c63b6fb2 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:04.691719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 443/873] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:42:04.697701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f2ee6-784c-45d9-9080-5568eb05d12a None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:04.698081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f2ee6-784c-45d9-9080-5568eb05d12a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:04.698463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f2ee6-784c-45d9-9080-5568eb05d12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:04.716442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-243f2ee6-784c-45d9-9080-5568eb05d12a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:04.717960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-243f2ee6-784c-45d9-9080-5568eb05d12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:04.757262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b8caa-8aee-4354-9675-d5dc381b09ef None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:04.794531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e38adb-cea2-43d5-981f-331dca88714d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:04.795346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e38adb-cea2-43d5-981f-331dca88714d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:04.795773 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e38adb-cea2-43d5-981f-331dca88714d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:04.796012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e38adb-cea2-43d5-981f-331dca88714d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:04.797139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e38adb-cea2-43d5-981f-331dca88714d None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=884ed6ddb71647088b33dc9623143b9c) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:04.797373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e38adb-cea2-43d5-981f-331dca88714d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:04.798874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e38adb-cea2-43d5-981f-331dca88714d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:04.807527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 444/874] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:04.813494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 431/875] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:04 2022] GET /identity/v3/role_assignments?user.id=884ed6ddb71647088b33dc9623143b9c&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:04.826285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f2ee6-784c-45d9-9080-5568eb05d12a None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:04.872520 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:04.939087 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f2ee6-784c-45d9-9080-5568eb05d12a None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:04.977336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d16c9d-fd11-489c-bd73-b99c0ed3d331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:04.978662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d16c9d-fd11-489c-bd73-b99c0ed3d331 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:04.979166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d16c9d-fd11-489c-bd73-b99c0ed3d331 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:04.979662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d16c9d-fd11-489c-bd73-b99c0ed3d331 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:04.980515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d16c9d-fd11-489c-bd73-b99c0ed3d331 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:04.982133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d16c9d-fd11-489c-bd73-b99c0ed3d331 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-08d16c9d-fd11-489c-bd73-b99c0ed3d331 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:04.994898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:04.996031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:04.996868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 445/876] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:05.002986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e38adb-cea2-43d5-981f-331dca88714d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:05.095028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:05.113776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e38adb-cea2-43d5-981f-331dca88714d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:05.146721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7386e-fc39-4bbf-9192-e6fbcb254a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:05.147421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7386e-fc39-4bbf-9192-e6fbcb254a23 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:05.147701 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7386e-fc39-4bbf-9192-e6fbcb254a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:05.147972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7386e-fc39-4bbf-9192-e6fbcb254a23 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:05.148682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7386e-fc39-4bbf-9192-e6fbcb254a23 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:05.148980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7386e-fc39-4bbf-9192-e6fbcb254a23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:05.150394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7386e-fc39-4bbf-9192-e6fbcb254a23 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:05.155455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 432/877] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 4 05:42:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:05.161044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d16c9d-fd11-489c-bd73-b99c0ed3d331 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:05.238821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59ee90-85d0-4158-a690-51a04c71a8cd None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:05.239169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59ee90-85d0-4158-a690-51a04c71a8cd None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:05.239399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59ee90-85d0-4158-a690-51a04c71a8cd None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:05.241072 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 433/878] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:05.246366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f248f-f761-4f6e-b013-cda708f6a97e None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:05.246920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f248f-f761-4f6e-b013-cda708f6a97e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:05.247509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f248f-f761-4f6e-b013-cda708f6a97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:05.264599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ec3f248f-f761-4f6e-b013-cda708f6a97e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:05.266203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ec3f248f-f761-4f6e-b013-cda708f6a97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:05.266948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d16c9d-fd11-489c-bd73-b99c0ed3d331 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:05.279614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:05.308547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84dbdf7-abae-4fe9-9331-0f5da372ee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:05.309226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84dbdf7-abae-4fe9-9331-0f5da372ee62 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:05.309541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84dbdf7-abae-4fe9-9331-0f5da372ee62 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:05.309827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84dbdf7-abae-4fe9-9331-0f5da372ee62 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:05.310545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84dbdf7-abae-4fe9-9331-0f5da372ee62 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:05.310850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84dbdf7-abae-4fe9-9331-0f5da372ee62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:05.312695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84dbdf7-abae-4fe9-9331-0f5da372ee62 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:05.318860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 446/879] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 4 05:42:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:05.347526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 434/880] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:05.371226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84dbdf7-abae-4fe9-9331-0f5da372ee62 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:05.500595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84dbdf7-abae-4fe9-9331-0f5da372ee62 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:05.535855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff3081-25d3-4650-9eea-f7037d91bbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:05.536531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3081-25d3-4650-9eea-f7037d91bbc2 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:05.536811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3081-25d3-4650-9eea-f7037d91bbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:05.537080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3081-25d3-4650-9eea-f7037d91bbc2 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:05.537818 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff3081-25d3-4650-9eea-f7037d91bbc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:05.539196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff3081-25d3-4650-9eea-f7037d91bbc2 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:05.559141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 447/881] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:42:05 2022] POST /identity/v3/services => generated 269 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:05.675489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdca9be-6832-4916-98cb-c659cf3bb49b None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:05.675897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdca9be-6832-4916-98cb-c659cf3bb49b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:05.676071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdca9be-6832-4916-98cb-c659cf3bb49b None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:05.677723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 435/882] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:05.683467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88c4a8-a792-44fc-a189-fc4fa3806af7 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:05.683779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88c4a8-a792-44fc-a189-fc4fa3806af7 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:05.684020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88c4a8-a792-44fc-a189-fc4fa3806af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:05.703907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-fc88c4a8-a792-44fc-a189-fc4fa3806af7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:05.705114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-fc88c4a8-a792-44fc-a189-fc4fa3806af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:05.727264 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:05.784768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 448/883] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:05.808741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdca9be-6832-4916-98cb-c659cf3bb49b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:05.895532 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:05.923420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdca9be-6832-4916-98cb-c659cf3bb49b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:05.961810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a3275-d760-4bf6-a96b-7769eec6e555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:05.963058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a3275-d760-4bf6-a96b-7769eec6e555 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:05.963535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a3275-d760-4bf6-a96b-7769eec6e555 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:05.964005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a3275-d760-4bf6-a96b-7769eec6e555 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:05.964824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a3275-d760-4bf6-a96b-7769eec6e555 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:05.966413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a3275-d760-4bf6-a96b-7769eec6e555 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-867a3275-d760-4bf6-a96b-7769eec6e555 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:05.979046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:05.980093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 05:42:05.980924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:05.983497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 436/884] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:42:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:05.989219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88c4a8-a792-44fc-a189-fc4fa3806af7 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:06.093292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88c4a8-a792-44fc-a189-fc4fa3806af7 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:06.133635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14cf27-9186-4fad-985b-36eb86631b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:06.134654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14cf27-9186-4fad-985b-36eb86631b8d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:06.135076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14cf27-9186-4fad-985b-36eb86631b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:06.135552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14cf27-9186-4fad-985b-36eb86631b8d None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:06.136402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14cf27-9186-4fad-985b-36eb86631b8d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:06.136865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14cf27-9186-4fad-985b-36eb86631b8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:06.138392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14cf27-9186-4fad-985b-36eb86631b8d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:06.143686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 449/885] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:42:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:06.152393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a3275-d760-4bf6-a96b-7769eec6e555 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:06.262905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a3275-d760-4bf6-a96b-7769eec6e555 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:06.297314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2185df3-95af-439d-b68e-fede0b93536d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:06.298244 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2185df3-95af-439d-b68e-fede0b93536d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:06.298681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2185df3-95af-439d-b68e-fede0b93536d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:06.299096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2185df3-95af-439d-b68e-fede0b93536d None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:06.299979 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2185df3-95af-439d-b68e-fede0b93536d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:06.300453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2185df3-95af-439d-b68e-fede0b93536d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:06.301962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2185df3-95af-439d-b68e-fede0b93536d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:06.307603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 437/886] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:42:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:06.313304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14cf27-9186-4fad-985b-36eb86631b8d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:06.332047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:06.417365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14cf27-9186-4fad-985b-36eb86631b8d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:06.456640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324d7ac-6568-4c53-90ee-aca451266038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:06.457897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324d7ac-6568-4c53-90ee-aca451266038 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:06.458310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324d7ac-6568-4c53-90ee-aca451266038 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:06.458709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324d7ac-6568-4c53-90ee-aca451266038 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:06.459744 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324d7ac-6568-4c53-90ee-aca451266038 None admin] RBAC: Adding query filter params (interface=public, service_id=05b9674aceb844ffaff8014f212614ec, region_id=RegionOne) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:06.460206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324d7ac-6568-4c53-90ee-aca451266038 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:06.461723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324d7ac-6568-4c53-90ee-aca451266038 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:06.475941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2185df3-95af-439d-b68e-fede0b93536d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:06.477022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 450/887] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 4 05:42:06 2022] GET /identity/v3/endpoints?service_id=05b9674aceb844ffaff8014f212614ec&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:06.479470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:06.594645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2185df3-95af-439d-b68e-fede0b93536d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:06.629282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0557cc-c44e-428d-9d1c-42d5478d4ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:06.630198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0557cc-c44e-428d-9d1c-42d5478d4ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:06.630626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0557cc-c44e-428d-9d1c-42d5478d4ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:06.631073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0557cc-c44e-428d-9d1c-42d5478d4ad4 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:06.631924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0557cc-c44e-428d-9d1c-42d5478d4ad4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:06.633445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0557cc-c44e-428d-9d1c-42d5478d4ad4 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:06.639777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 438/888] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 4 05:42:06 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:42:07.049190 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff0fa0-e1e4-405a-9635-b26e3b3f716f None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.049759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff0fa0-e1e4-405a-9635-b26e3b3f716f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.049759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff0fa0-e1e4-405a-9635-b26e3b3f716f None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.051425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 451/889] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:07.056755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f491806-715d-4d49-b3e3-050324533348 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.057046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f491806-715d-4d49-b3e3-050324533348 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.057323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f491806-715d-4d49-b3e3-050324533348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.074842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4f491806-715d-4d49-b3e3-050324533348 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:07.076067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4f491806-715d-4d49-b3e3-050324533348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:07.143671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:07.152503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 439/890] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:07.176617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff0fa0-e1e4-405a-9635-b26e3b3f716f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:07.262840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:07.282908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff0fa0-e1e4-405a-9635-b26e3b3f716f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:07.316624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71a348-56e1-400a-a0fa-abe666db943a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:07.317363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a348-56e1-400a-a0fa-abe666db943a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.317637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a348-56e1-400a-a0fa-abe666db943a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.317902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a348-56e1-400a-a0fa-abe666db943a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.329511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71a348-56e1-400a-a0fa-abe666db943a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-af71a348-56e1-400a-a0fa-abe666db943a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:07.337946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:07.338897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:07.339779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:07.343361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 452/891] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 4 05:42:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:07.345613 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f491806-715d-4d49-b3e3-050324533348 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:07.386553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26089d55-b70b-4ffc-b525-5d58221163bd None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.386916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26089d55-b70b-4ffc-b525-5d58221163bd None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.387154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26089d55-b70b-4ffc-b525-5d58221163bd None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.388858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 453/892] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:42:07.393831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4e91f-f675-4829-8c3a-71e7f539e084 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.394125 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4e91f-f675-4829-8c3a-71e7f539e084 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.394397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4e91f-f675-4829-8c3a-71e7f539e084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.411528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-e1c4e91f-f675-4829-8c3a-71e7f539e084 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:07.412736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e1c4e91f-f675-4829-8c3a-71e7f539e084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:07.467224 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f491806-715d-4d49-b3e3-050324533348 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:07.487580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 454/893] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:07.504952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49512ea-d86c-4b3c-baf7-29777ff15be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:07.505956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4e91f-f675-4829-8c3a-71e7f539e084 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:07.506167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49512ea-d86c-4b3c-baf7-29777ff15be9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.506671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49512ea-d86c-4b3c-baf7-29777ff15be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.507152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49512ea-d86c-4b3c-baf7-29777ff15be9 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.508197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49512ea-d86c-4b3c-baf7-29777ff15be9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:07.508858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49512ea-d86c-4b3c-baf7-29777ff15be9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:07.511465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49512ea-d86c-4b3c-baf7-29777ff15be9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:07.527424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 440/894] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 4 05:42:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:07.528891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49512ea-d86c-4b3c-baf7-29777ff15be9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:07.595097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:07.596129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:07.635392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4e91f-f675-4829-8c3a-71e7f539e084 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:07.635587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49512ea-d86c-4b3c-baf7-29777ff15be9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:07.675534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d392d-f1c0-478e-a6f7-eb8584cb836b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:07.676287 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d392d-f1c0-478e-a6f7-eb8584cb836b None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.676571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d392d-f1c0-478e-a6f7-eb8584cb836b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.676873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d392d-f1c0-478e-a6f7-eb8584cb836b None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.677604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d392d-f1c0-478e-a6f7-eb8584cb836b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:07.678431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5588568-d87c-484c-acb0-b267d3fea5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:07.678970 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d392d-f1c0-478e-a6f7-eb8584cb836b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:07.679708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5588568-d87c-484c-acb0-b267d3fea5ec None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.680186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5588568-d87c-484c-acb0-b267d3fea5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.680668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5588568-d87c-484c-acb0-b267d3fea5ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.693055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5588568-d87c-484c-acb0-b267d3fea5ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:07.695293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5588568-d87c-484c-acb0-b267d3fea5ec None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:07.703662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 455/895] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:42:07 2022] POST /identity/v3/services => generated 241 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-d5588568-d87c-484c-acb0-b267d3fea5ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:07.714382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:07.715663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:07.716557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:07.716557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:07.716557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:07.716557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:07.718897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 441/896] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:07.725047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d392d-f1c0-478e-a6f7-eb8584cb836b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:07.836801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66606c2-4c16-4b1d-9562-dc5ea963fc83 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.837152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66606c2-4c16-4b1d-9562-dc5ea963fc83 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.837430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66606c2-4c16-4b1d-9562-dc5ea963fc83 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.843235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 442/897] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:07.844744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672e6b8-9464-471c-aacb-2d8667cadde8 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.845092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672e6b8-9464-471c-aacb-2d8667cadde8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.845420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672e6b8-9464-471c-aacb-2d8667cadde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.862583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d392d-f1c0-478e-a6f7-eb8584cb836b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:07.863140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-6672e6b8-9464-471c-aacb-2d8667cadde8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:07.864375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6672e6b8-9464-471c-aacb-2d8667cadde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:07.894081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:07.907982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdd426-948e-4e95-bdb7-67275125c67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:07.908970 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdd426-948e-4e95-bdb7-67275125c67f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:07.909251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdd426-948e-4e95-bdb7-67275125c67f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:07.909524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdd426-948e-4e95-bdb7-67275125c67f None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:07.910256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdd426-948e-4e95-bdb7-67275125c67f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:07.910561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdd426-948e-4e95-bdb7-67275125c67f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:07.912381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdd426-948e-4e95-bdb7-67275125c67f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:07.925862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 456/898] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:07.931690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdd426-948e-4e95-bdb7-67275125c67f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:07.954791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 443/899] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:07.977137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672e6b8-9464-471c-aacb-2d8667cadde8 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:08.034792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:08.045453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdd426-948e-4e95-bdb7-67275125c67f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:08.093260 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7db9a0-09e3-415a-972d-d77520d013e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:08.095536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7db9a0-09e3-415a-972d-d77520d013e7 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.096026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7db9a0-09e3-415a-972d-d77520d013e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.096749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7db9a0-09e3-415a-972d-d77520d013e7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.107556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7db9a0-09e3-415a-972d-d77520d013e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:08.107733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672e6b8-9464-471c-aacb-2d8667cadde8 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:08.109689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7db9a0-09e3-415a-972d-d77520d013e7 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-7a7db9a0-09e3-415a-972d-d77520d013e7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:42:08.130557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:08.131616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:08.132598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:08.132598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:08.132598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:42:08.132598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:42:08.132598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:42:08.132598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:42:08.132598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:42:08.132598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:42:08.132598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:42:08.132598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:08.134056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 457/900] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 4 05:42:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:08.138445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7db9a0-09e3-415a-972d-d77520d013e7 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:08.147500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c09e7d-cdf1-4996-98a5-76d8acd61530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:08.148342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c09e7d-cdf1-4996-98a5-76d8acd61530 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.148624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c09e7d-cdf1-4996-98a5-76d8acd61530 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.148900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c09e7d-cdf1-4996-98a5-76d8acd61530 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.149562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c09e7d-cdf1-4996-98a5-76d8acd61530 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:08.150977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c09e7d-cdf1-4996-98a5-76d8acd61530 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-52c09e7d-cdf1-4996-98a5-76d8acd61530 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:08.163667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:08.165060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 05:42:08.166664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:08.170375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 444/901] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Jul 4 05:42:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:08.171306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c09e7d-cdf1-4996-98a5-76d8acd61530 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:08.204201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:08.269242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7db9a0-09e3-415a-972d-d77520d013e7 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:08.294460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c09e7d-cdf1-4996-98a5-76d8acd61530 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:08.298285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:08.306009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ce679-6c70-445e-bee8-ce54b0d48deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:08.306795 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ce679-6c70-445e-bee8-ce54b0d48deb None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.307328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ce679-6c70-445e-bee8-ce54b0d48deb None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.307679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ce679-6c70-445e-bee8-ce54b0d48deb None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.308660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ce679-6c70-445e-bee8-ce54b0d48deb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:08.309176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ce679-6c70-445e-bee8-ce54b0d48deb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:08.310824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ce679-6c70-445e-bee8-ce54b0d48deb None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:08.330665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 458/902] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 4 05:42:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:08.335760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ce679-6c70-445e-bee8-ce54b0d48deb None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:08.339152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0911b-de52-4879-a9e9-562080875462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:08.340882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0911b-de52-4879-a9e9-562080875462 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.341302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0911b-de52-4879-a9e9-562080875462 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.341706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0911b-de52-4879-a9e9-562080875462 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.342571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0911b-de52-4879-a9e9-562080875462 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:08.343005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0911b-de52-4879-a9e9-562080875462 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:08.345044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0911b-de52-4879-a9e9-562080875462 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:08.352651 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 445/903] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:08.357745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0911b-de52-4879-a9e9-562080875462 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:08.447460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ce679-6c70-445e-bee8-ce54b0d48deb None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:08.450207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:08.481694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0911b-de52-4879-a9e9-562080875462 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:08.490474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382915c-67f3-49f8-9942-822cebaa028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:08.491503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382915c-67f3-49f8-9942-822cebaa028a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.491982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382915c-67f3-49f8-9942-822cebaa028a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.492475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382915c-67f3-49f8-9942-822cebaa028a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.518414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb227765-faf8-4749-bc50-3c71bc8181a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:08.519101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb227765-faf8-4749-bc50-3c71bc8181a7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.519408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb227765-faf8-4749-bc50-3c71bc8181a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.519679 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb227765-faf8-4749-bc50-3c71bc8181a7 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.520654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb227765-faf8-4749-bc50-3c71bc8181a7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:08.520654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb227765-faf8-4749-bc50-3c71bc8181a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:08.522171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb227765-faf8-4749-bc50-3c71bc8181a7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:08.532824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 446/904] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:08.533743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb227765-faf8-4749-bc50-3c71bc8181a7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:08.566325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:08.567455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382915c-67f3-49f8-9942-822cebaa028a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:08.568852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382915c-67f3-49f8-9942-822cebaa028a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:08.642190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb227765-faf8-4749-bc50-3c71bc8181a7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:08.675862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 459/905] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:08 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:42:08.676995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382915c-67f3-49f8-9942-822cebaa028a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:08.681115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:08.684479 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f4676-4aa5-4b03-a4c2-132a17a40ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:08.685158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f4676-4aa5-4b03-a4c2-132a17a40ebd None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.685436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f4676-4aa5-4b03-a4c2-132a17a40ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.685707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f4676-4aa5-4b03-a4c2-132a17a40ebd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.686507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f4676-4aa5-4b03-a4c2-132a17a40ebd None admin] RBAC: Adding query filter params (interface=public, service_id=820f85eed87647058f98cff198a73f00, region_id=RegionOne) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:08.686812 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f4676-4aa5-4b03-a4c2-132a17a40ebd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:08.688221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f4676-4aa5-4b03-a4c2-132a17a40ebd None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:08.698918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 447/906] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 4 05:42:08 2022] GET /identity/v3/endpoints?service_id=820f85eed87647058f98cff198a73f00&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:08.699795 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f4676-4aa5-4b03-a4c2-132a17a40ebd None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:08.788538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382915c-67f3-49f8-9942-822cebaa028a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:08.798046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:08.804965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f4676-4aa5-4b03-a4c2-132a17a40ebd None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:08.826737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a807d0e-78a2-48de-aa1c-85d8b5cdf416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:08.827715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a807d0e-78a2-48de-aa1c-85d8b5cdf416 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.827986 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a807d0e-78a2-48de-aa1c-85d8b5cdf416 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.828327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a807d0e-78a2-48de-aa1c-85d8b5cdf416 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.839783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a807d0e-78a2-48de-aa1c-85d8b5cdf416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:08.841349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a807d0e-78a2-48de-aa1c-85d8b5cdf416 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:08.842184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae766c4-a4b2-4b83-9be6-89b36d5bc1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:08.843399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae766c4-a4b2-4b83-9be6-89b36d5bc1ff None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.843918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae766c4-a4b2-4b83-9be6-89b36d5bc1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.844397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae766c4-a4b2-4b83-9be6-89b36d5bc1ff None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.845174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae766c4-a4b2-4b83-9be6-89b36d5bc1ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:08.846549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae766c4-a4b2-4b83-9be6-89b36d5bc1ff None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:08.861171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 448/907] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 4 05:42:08 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:42:08.861171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c198ae-964b-4012-b596-fa9f6c00c8d3 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.861171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c198ae-964b-4012-b596-fa9f6c00c8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.861171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c198ae-964b-4012-b596-fa9f6c00c8d3 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.861171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 449/908] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-6a807d0e-78a2-48de-aa1c-85d8b5cdf416 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:08.861635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:08.862433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:08.863386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:08.863386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:08.863386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:08.863386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:08.863386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 460/909] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:08.865147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffe5e1-e19b-48a6-bb5a-249c5ac2d841 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:08.865147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffe5e1-e19b-48a6-bb5a-249c5ac2d841 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:08.865362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffe5e1-e19b-48a6-bb5a-249c5ac2d841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:08.867528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a807d0e-78a2-48de-aa1c-85d8b5cdf416 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:08.883495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-aaffe5e1-e19b-48a6-bb5a-249c5ac2d841 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:08.885005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-aaffe5e1-e19b-48a6-bb5a-249c5ac2d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:08.934276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:08.967276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 450/910] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:08.984777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffe5e1-e19b-48a6-bb5a-249c5ac2d841 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:08.996503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a807d0e-78a2-48de-aa1c-85d8b5cdf416 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:09.035240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a60bd-ff1d-4e61-8781-322f4d6688e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:09.036355 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a60bd-ff1d-4e61-8781-322f4d6688e1 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:09.036930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a60bd-ff1d-4e61-8781-322f4d6688e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:09.037396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a60bd-ff1d-4e61-8781-322f4d6688e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:09.038377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a60bd-ff1d-4e61-8781-322f4d6688e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:09.038878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a60bd-ff1d-4e61-8781-322f4d6688e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:09.040599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a60bd-ff1d-4e61-8781-322f4d6688e1 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:09.060851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 461/911] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:09.063329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a60bd-ff1d-4e61-8781-322f4d6688e1 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:09.083395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:09.093304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffe5e1-e19b-48a6-bb5a-249c5ac2d841 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:09.135589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3ee01-9a2e-4128-90fa-342f0782f9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:09.136672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3ee01-9a2e-4128-90fa-342f0782f9bb None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:09.136948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3ee01-9a2e-4128-90fa-342f0782f9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:09.137504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3ee01-9a2e-4128-90fa-342f0782f9bb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:09.138563 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3ee01-9a2e-4128-90fa-342f0782f9bb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:09.140460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3ee01-9a2e-4128-90fa-342f0782f9bb None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-2ac3ee01-9a2e-4128-90fa-342f0782f9bb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:09.154386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:09.155481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:09.156313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 451/912] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:42:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:09.163676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3ee01-9a2e-4128-90fa-342f0782f9bb None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:09.192029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a60bd-ff1d-4e61-8781-322f4d6688e1 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:09.213789 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:09.236403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3d8f0-e959-4e21-b92e-e45901a4e1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:09.237448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3d8f0-e959-4e21-b92e-e45901a4e1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:09.237736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3d8f0-e959-4e21-b92e-e45901a4e1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:09.238008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3d8f0-e959-4e21-b92e-e45901a4e1e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:09.248307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3d8f0-e959-4e21-b92e-e45901a4e1e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:09.249842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3d8f0-e959-4e21-b92e-e45901a4e1e6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-f8b3d8f0-e959-4e21-b92e-e45901a4e1e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:42:09.271100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:09.272104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:09.272987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:42:09.272987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:42:09.272987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:42:09.272987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:42:09.272987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:42:09.272987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:42:09.272987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:09.272987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:09.275322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 462/913] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:09.278285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3d8f0-e959-4e21-b92e-e45901a4e1e6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:09.284716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3ee01-9a2e-4128-90fa-342f0782f9bb None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:09.328122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb2e8-94c9-4b86-86b6-14757113633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:09.328928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb2e8-94c9-4b86-86b6-14757113633b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:09.329208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb2e8-94c9-4b86-86b6-14757113633b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:09.329482 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb2e8-94c9-4b86-86b6-14757113633b None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:09.330193 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb2e8-94c9-4b86-86b6-14757113633b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:09.330490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb2e8-94c9-4b86-86b6-14757113633b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:09.332340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb2e8-94c9-4b86-86b6-14757113633b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:09.338495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 452/914] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:42:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:09.343999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb2e8-94c9-4b86-86b6-14757113633b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:09.364938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:09.395774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3d8f0-e959-4e21-b92e-e45901a4e1e6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:09.431617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c67ffd-e5bb-412a-aaeb-5ee234f78f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:09.432290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c67ffd-e5bb-412a-aaeb-5ee234f78f6c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:09.432567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c67ffd-e5bb-412a-aaeb-5ee234f78f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:09.432836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c67ffd-e5bb-412a-aaeb-5ee234f78f6c None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:09.433629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c67ffd-e5bb-412a-aaeb-5ee234f78f6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:09.433934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c67ffd-e5bb-412a-aaeb-5ee234f78f6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:09.435799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c67ffd-e5bb-412a-aaeb-5ee234f78f6c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:09.453054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 463/915] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:09.455336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c67ffd-e5bb-412a-aaeb-5ee234f78f6c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:09.466533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb2e8-94c9-4b86-86b6-14757113633b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:09.502799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:09.512405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370b228-c390-4bad-b0c5-60b0461d1e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:09.513256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370b228-c390-4bad-b0c5-60b0461d1e84 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:09.513637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370b228-c390-4bad-b0c5-60b0461d1e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:09.514402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370b228-c390-4bad-b0c5-60b0461d1e84 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:09.515521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370b228-c390-4bad-b0c5-60b0461d1e84 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:09.515933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370b228-c390-4bad-b0c5-60b0461d1e84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:09.517694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370b228-c390-4bad-b0c5-60b0461d1e84 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:09.527326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 453/916] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 4 05:42:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:09.530282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370b228-c390-4bad-b0c5-60b0461d1e84 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:09.588282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c67ffd-e5bb-412a-aaeb-5ee234f78f6c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:09.630060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821c9e1-d016-469c-a1b4-4ce64af434dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:09.631337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821c9e1-d016-469c-a1b4-4ce64af434dd None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:09.632016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821c9e1-d016-469c-a1b4-4ce64af434dd None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:09.632432 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821c9e1-d016-469c-a1b4-4ce64af434dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:09.633591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821c9e1-d016-469c-a1b4-4ce64af434dd None admin] RBAC: Adding query filter params (role.id=85e17d8dc6cb480abe4e2bb231995a79, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=884ed6ddb71647088b33dc9623143b9c) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:09.634092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821c9e1-d016-469c-a1b4-4ce64af434dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:09.634731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:09.636780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821c9e1-d016-469c-a1b4-4ce64af434dd None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:09.651695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 464/917] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:09 2022] GET /identity/v3/role_assignments?user.id=884ed6ddb71647088b33dc9623143b9c&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=85e17d8dc6cb480abe4e2bb231995a79 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:09.664020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370b228-c390-4bad-b0c5-60b0461d1e84 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:09.702689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ff9c-f9c0-4b6d-ad63-f1e7cb0d6d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:09.703469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ff9c-f9c0-4b6d-ad63-f1e7cb0d6d3e None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:09.703696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ff9c-f9c0-4b6d-ad63-f1e7cb0d6d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:09.703966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ff9c-f9c0-4b6d-ad63-f1e7cb0d6d3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:09.704720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385ff9c-f9c0-4b6d-ad63-f1e7cb0d6d3e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:09.706285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385ff9c-f9c0-4b6d-ad63-f1e7cb0d6d3e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:09.731066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 454/918] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 4 05:42:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:42:09.847290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39010441-9c78-4fa9-937d-0314482c42b1 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:09.847621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39010441-9c78-4fa9-937d-0314482c42b1 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:09.847843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39010441-9c78-4fa9-937d-0314482c42b1 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:09.849521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 465/919] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:09.855987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67e8f8-fd3b-4dd0-ad66-c32045e0b463 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:09.856292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67e8f8-fd3b-4dd0-ad66-c32045e0b463 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:09.856627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67e8f8-fd3b-4dd0-ad66-c32045e0b463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:09.875489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ee67e8f8-fd3b-4dd0-ad66-c32045e0b463 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:09.876722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ee67e8f8-fd3b-4dd0-ad66-c32045e0b463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:09.958741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 455/920] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:09.982270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39010441-9c78-4fa9-937d-0314482c42b1 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:10.086493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:10.095831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39010441-9c78-4fa9-937d-0314482c42b1 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:10.131278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c42f4a-7a81-4cd4-9131-95d1e13e495f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:10.132126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c42f4a-7a81-4cd4-9131-95d1e13e495f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:10.132406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c42f4a-7a81-4cd4-9131-95d1e13e495f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:10.132675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c42f4a-7a81-4cd4-9131-95d1e13e495f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:10.133329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c42f4a-7a81-4cd4-9131-95d1e13e495f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:10.134758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c42f4a-7a81-4cd4-9131-95d1e13e495f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-b1c42f4a-7a81-4cd4-9131-95d1e13e495f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:10.147510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:10.148571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:10.149398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:10.153808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 466/921] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:10.155183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67e8f8-fd3b-4dd0-ad66-c32045e0b463 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:10.219644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:10.261791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67e8f8-fd3b-4dd0-ad66-c32045e0b463 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:10.300056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b07fc-6fd8-44b5-8317-ada0bead7475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:10.300822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b07fc-6fd8-44b5-8317-ada0bead7475 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:10.301167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b07fc-6fd8-44b5-8317-ada0bead7475 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:10.301439 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b07fc-6fd8-44b5-8317-ada0bead7475 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:10.302148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b07fc-6fd8-44b5-8317-ada0bead7475 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:10.302448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b07fc-6fd8-44b5-8317-ada0bead7475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:10.303963 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b07fc-6fd8-44b5-8317-ada0bead7475 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:10.309799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 456/922] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 4 05:42:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:10.315734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c42f4a-7a81-4cd4-9131-95d1e13e495f None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:10.419132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c42f4a-7a81-4cd4-9131-95d1e13e495f None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:10.454436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443ad35-84cf-49d4-8f5e-1b679e48b469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:10.455108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443ad35-84cf-49d4-8f5e-1b679e48b469 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:10.455417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443ad35-84cf-49d4-8f5e-1b679e48b469 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:10.455693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443ad35-84cf-49d4-8f5e-1b679e48b469 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:10.456399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443ad35-84cf-49d4-8f5e-1b679e48b469 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:10.456703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443ad35-84cf-49d4-8f5e-1b679e48b469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:10.458088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443ad35-84cf-49d4-8f5e-1b679e48b469 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:10.463514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 467/923] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 4 05:42:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:10.472433 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b07fc-6fd8-44b5-8317-ada0bead7475 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:10.579618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b07fc-6fd8-44b5-8317-ada0bead7475 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:10.605693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:10.614500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd574af-d0a3-46ac-a896-1ec18b22c866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:10.615177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd574af-d0a3-46ac-a896-1ec18b22c866 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:10.615488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd574af-d0a3-46ac-a896-1ec18b22c866 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:10.615762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd574af-d0a3-46ac-a896-1ec18b22c866 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:10.616558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd574af-d0a3-46ac-a896-1ec18b22c866 None admin] RBAC: Adding query filter params (interface=public, service_id=94636d724a8a41a6b90b664210e01442, region_id=RegionOne) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:10.616867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd574af-d0a3-46ac-a896-1ec18b22c866 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:10.618246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd574af-d0a3-46ac-a896-1ec18b22c866 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:10.628788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 457/924] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 4 05:42:10 2022] GET /identity/v3/endpoints?service_id=94636d724a8a41a6b90b664210e01442&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:10.630246 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443ad35-84cf-49d4-8f5e-1b679e48b469 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:10.660243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:10.733804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443ad35-84cf-49d4-8f5e-1b679e48b469 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:10.767759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8e6b5-f5e9-4b65-9f62-d7adc22ef71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:10.768435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8e6b5-f5e9-4b65-9f62-d7adc22ef71c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:10.768713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8e6b5-f5e9-4b65-9f62-d7adc22ef71c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:10.768986 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8e6b5-f5e9-4b65-9f62-d7adc22ef71c None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:10.769664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8e6b5-f5e9-4b65-9f62-d7adc22ef71c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:10.771117 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8e6b5-f5e9-4b65-9f62-d7adc22ef71c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:10.783412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 468/925] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 4 05:42:10 2022] GET /identity/v3/services => generated 1497 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:42:11.041389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149d9e2-62f1-459c-8e71-5ebcaa86057d None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:11.041781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149d9e2-62f1-459c-8e71-5ebcaa86057d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:11.041949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149d9e2-62f1-459c-8e71-5ebcaa86057d None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:11.043618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 458/926] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:11.049185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2482335-227b-483a-8ae1-d7b486225366 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:11.049479 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2482335-227b-483a-8ae1-d7b486225366 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:11.049760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2482335-227b-483a-8ae1-d7b486225366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:11.067124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-e2482335-227b-483a-8ae1-d7b486225366 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:11.068359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e2482335-227b-483a-8ae1-d7b486225366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:11.141089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 469/927] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:11.163754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149d9e2-62f1-459c-8e71-5ebcaa86057d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:11.273808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149d9e2-62f1-459c-8e71-5ebcaa86057d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:11.368094 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfd959-88ce-4d40-afdd-fb4622c71324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:11.368872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfd959-88ce-4d40-afdd-fb4622c71324 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:11.369151 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfd959-88ce-4d40-afdd-fb4622c71324 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:11.369420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfd959-88ce-4d40-afdd-fb4622c71324 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:11.370078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfd959-88ce-4d40-afdd-fb4622c71324 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:11.371551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfd959-88ce-4d40-afdd-fb4622c71324 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-30bfd959-88ce-4d40-afdd-fb4622c71324 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:11.386100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:11.388212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:11.390143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 459/928] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Jul 4 05:42:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:11.399276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2482335-227b-483a-8ae1-d7b486225366 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:11.455633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:11.551174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2482335-227b-483a-8ae1-d7b486225366 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:11.587766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a428796-63ef-4824-91d1-e31f21299ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:11.588563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a428796-63ef-4824-91d1-e31f21299ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:11.588845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a428796-63ef-4824-91d1-e31f21299ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:11.589119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a428796-63ef-4824-91d1-e31f21299ef6 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:11.589820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a428796-63ef-4824-91d1-e31f21299ef6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:11.590122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a428796-63ef-4824-91d1-e31f21299ef6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:11.591781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a428796-63ef-4824-91d1-e31f21299ef6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:11.597755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 470/929] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:11.604606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfd959-88ce-4d40-afdd-fb4622c71324 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:11.687895 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:11.740970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfd959-88ce-4d40-afdd-fb4622c71324 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:11.762820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a1cd4-7b9d-4dc0-b48e-2fc153404915 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:11.763366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a1cd4-7b9d-4dc0-b48e-2fc153404915 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:11.763366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a1cd4-7b9d-4dc0-b48e-2fc153404915 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:11.771369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 471/930] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:11.781019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75187d1c-5d6b-474f-9fbd-30849dcab60f None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:11.781301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75187d1c-5d6b-474f-9fbd-30849dcab60f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:11.781619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75187d1c-5d6b-474f-9fbd-30849dcab60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:11.791063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c055f28-fd4c-4da4-a89b-030def592487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:11.792039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c055f28-fd4c-4da4-a89b-030def592487 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:11.793368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c055f28-fd4c-4da4-a89b-030def592487 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:11.793835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c055f28-fd4c-4da4-a89b-030def592487 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:11.794804 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c055f28-fd4c-4da4-a89b-030def592487 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:11.795342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c055f28-fd4c-4da4-a89b-030def592487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:11.796909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c055f28-fd4c-4da4-a89b-030def592487 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:11.809321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 460/931] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:11.814454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c055f28-fd4c-4da4-a89b-030def592487 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:11.817278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-75187d1c-5d6b-474f-9fbd-30849dcab60f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:11.818726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-75187d1c-5d6b-474f-9fbd-30849dcab60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:11.935282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 472/932] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:11.949470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c055f28-fd4c-4da4-a89b-030def592487 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:11.951076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75187d1c-5d6b-474f-9fbd-30849dcab60f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:11.977728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:11.989402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:12.004761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24362a3-1fbf-49f4-95ab-ea889a1493eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:12.005590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24362a3-1fbf-49f4-95ab-ea889a1493eb None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.006086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24362a3-1fbf-49f4-95ab-ea889a1493eb None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.006618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24362a3-1fbf-49f4-95ab-ea889a1493eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.007460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24362a3-1fbf-49f4-95ab-ea889a1493eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:12.009036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24362a3-1fbf-49f4-95ab-ea889a1493eb None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:12.035698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 461/933] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 4 05:42:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:12.081348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75187d1c-5d6b-474f-9fbd-30849dcab60f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:12.128758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b079c86-5967-409c-ba0f-2c515e6bfa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:12.129674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b079c86-5967-409c-ba0f-2c515e6bfa4b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.130000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b079c86-5967-409c-ba0f-2c515e6bfa4b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.130277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b079c86-5967-409c-ba0f-2c515e6bfa4b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.142730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b079c86-5967-409c-ba0f-2c515e6bfa4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-1b079c86-5967-409c-ba0f-2c515e6bfa4b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:12.151635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:12.152708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:12.153608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 473/934] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 4 05:42:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:12.159564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97054715-abc7-46db-a182-dc8f66e6ad7f None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.159904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97054715-abc7-46db-a182-dc8f66e6ad7f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.160185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97054715-abc7-46db-a182-dc8f66e6ad7f None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.162188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b079c86-5967-409c-ba0f-2c515e6bfa4b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:12.163395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 462/935] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:12.170150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ba8b8-402c-4cf6-8511-e27635cbccd2 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.170474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ba8b8-402c-4cf6-8511-e27635cbccd2 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.170715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ba8b8-402c-4cf6-8511-e27635cbccd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.191826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-989ba8b8-402c-4cf6-8511-e27635cbccd2 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:12.193026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-989ba8b8-402c-4cf6-8511-e27635cbccd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:12.244318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:12.272801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 463/936] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:12.281222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b079c86-5967-409c-ba0f-2c515e6bfa4b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:12.296010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ba8b8-402c-4cf6-8511-e27635cbccd2 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:12.320569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb9086-40f9-4c43-a931-469fd758a9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:12.321544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb9086-40f9-4c43-a931-469fd758a9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.322031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb9086-40f9-4c43-a931-469fd758a9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.322460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb9086-40f9-4c43-a931-469fd758a9d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.323937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb9086-40f9-4c43-a931-469fd758a9d2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:12.324399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb9086-40f9-4c43-a931-469fd758a9d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:12.326229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb9086-40f9-4c43-a931-469fd758a9d2 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:12.338487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 474/937] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 4 05:42:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:12.343733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb9086-40f9-4c43-a931-469fd758a9d2 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:12.415276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ba8b8-402c-4cf6-8511-e27635cbccd2 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:12.424004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:12.461551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:12.462582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.463020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.463480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.464360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:12.466857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:12.469237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: WARNING keystone.server.flask.common [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 04 05:42:12.477648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb9086-40f9-4c43-a931-469fd758a9d2 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:12.493973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.sql.core [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 05:42:12.493973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 05:42:12.493973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [parameters: {'id': 'dabcb9195cd148a2ad4c4975c987793b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 05:42:12.493973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80129) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 05:42:12.520712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e455e9-759e-4c9d-a581-25eff6e27a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:12.521594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e455e9-759e-4c9d-a581-25eff6e27a56 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.521983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e455e9-759e-4c9d-a581-25eff6e27a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.522357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e455e9-759e-4c9d-a581-25eff6e27a56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.524712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dabcb9195cd148a2ad4c4975c987793b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dabcb9195cd148a2ad4c4975c987793b. Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application conn.query(q) Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 05:42:12.526623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application result.read() Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 04 05:42:12.527683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application next(self.gen) Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application yield resource Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application next(self.gen) Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise self.value Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application session.commit() Jul 04 05:42:12.528926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self.session.flush() Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application compat.raise_( Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 05:42:12.530040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 05:42:12.532111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application conn.query(q) Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application result.read() Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 05:42:12.533320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [parameters: {'id': 'dabcb9195cd148a2ad4c4975c987793b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:12.534467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:12.535484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:12.536605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:12.536605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 04 05:42:12.536605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 04 05:42:12.536605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:12.536605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:12.536605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 04 05:42:12.536605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 04 05:42:12.536605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dabcb9195cd148a2ad4c4975c987793b. Jul 04 05:42:12.536605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:12.539307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 464/938] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Jul 4 05:42:12 2022] POST /identity/v3/projects => generated 262 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:12.546311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:12.550254 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e455e9-759e-4c9d-a581-25eff6e27a56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:12.552131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e455e9-759e-4c9d-a581-25eff6e27a56 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-c6e455e9-759e-4c9d-a581-25eff6e27a56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:12.576277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:12.577297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:12.578543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:12.578543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:12.578543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:12.578543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:12.578543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 475/939] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:12.583078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e455e9-759e-4c9d-a581-25eff6e27a56 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:12.685660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d585985-003b-4866-b49f-f3b5d7d4789b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:12.696125 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e455e9-759e-4c9d-a581-25eff6e27a56 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:12.720472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32e09f-f8fc-46bf-b964-b4b274f743e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:12.721468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32e09f-f8fc-46bf-b964-b4b274f743e8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.721852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32e09f-f8fc-46bf-b964-b4b274f743e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.722224 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32e09f-f8fc-46bf-b964-b4b274f743e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.729306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32e09f-f8fc-46bf-b964-b4b274f743e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:12.731185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32e09f-f8fc-46bf-b964-b4b274f743e8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:12.734194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280d8f9-52ce-4d48-9c58-60db33322991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:12.735079 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280d8f9-52ce-4d48-9c58-60db33322991 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.735531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280d8f9-52ce-4d48-9c58-60db33322991 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.735964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280d8f9-52ce-4d48-9c58-60db33322991 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.736893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280d8f9-52ce-4d48-9c58-60db33322991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:12.737405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280d8f9-52ce-4d48-9c58-60db33322991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:12.738964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280d8f9-52ce-4d48-9c58-60db33322991 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-fd32e09f-f8fc-46bf-b964-b4b274f743e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:42:12.747121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:12.748105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:12.749005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:42:12.749005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:42:12.749005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:42:12.749005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:42:12.749005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:42:12.749005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:42:12.749005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:12.749005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:12.752044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 465/940] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:12.754237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32e09f-f8fc-46bf-b964-b4b274f743e8 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:12.755969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 476/941] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:12.765050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280d8f9-52ce-4d48-9c58-60db33322991 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:12.780231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:12.819841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:12.860464 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32e09f-f8fc-46bf-b964-b4b274f743e8 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:12.877412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280d8f9-52ce-4d48-9c58-60db33322991 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:12.896723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb98c8-fd63-43ee-99cd-ff8fff225e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:12.897483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb98c8-fd63-43ee-99cd-ff8fff225e96 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.897826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb98c8-fd63-43ee-99cd-ff8fff225e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.898163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb98c8-fd63-43ee-99cd-ff8fff225e96 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.898987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb98c8-fd63-43ee-99cd-ff8fff225e96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:12.899386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb98c8-fd63-43ee-99cd-ff8fff225e96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:12.901305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb98c8-fd63-43ee-99cd-ff8fff225e96 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:12.915038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41375312-a5cf-426c-89d8-d51cab8d9e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:12.916216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41375312-a5cf-426c-89d8-d51cab8d9e73 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.916668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41375312-a5cf-426c-89d8-d51cab8d9e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.917138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41375312-a5cf-426c-89d8-d51cab8d9e73 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.925460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41375312-a5cf-426c-89d8-d51cab8d9e73 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:12.927205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41375312-a5cf-426c-89d8-d51cab8d9e73 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:12.929591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 466/942] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:42:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:12.932909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f74c-0849-4aae-bbec-1e05d53cbd30 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.933196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f74c-0849-4aae-bbec-1e05d53cbd30 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.933465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f74c-0849-4aae-bbec-1e05d53cbd30 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.934952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 467/943] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:12.940709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093180-1486-4e34-93b9-ade4814a4a76 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:12.940989 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093180-1486-4e34-93b9-ade4814a4a76 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:12.941258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093180-1486-4e34-93b9-ade4814a4a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-41375312-a5cf-426c-89d8-d51cab8d9e73 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:42:12.944765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:12.945693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:12.946890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:12.946890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:12.946890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:42:12.946890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:42:12.946890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:42:12.946890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:42:12.946890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:42:12.946890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:42:12.946890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:42:12.946890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:12.951054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 477/944] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 4 05:42:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:12.952579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41375312-a5cf-426c-89d8-d51cab8d9e73 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:12.965836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-60093180-1486-4e34-93b9-ade4814a4a76 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:12.967526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-60093180-1486-4e34-93b9-ade4814a4a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:13.043878 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 468/945] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:13.063712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41375312-a5cf-426c-89d8-d51cab8d9e73 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:13.067842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60093180-1486-4e34-93b9-ade4814a4a76 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:13.072953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:13.088352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:13.101386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cb56b-25a6-49b6-b7de-59b93e307a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:13.102070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cb56b-25a6-49b6-b7de-59b93e307a2c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:13.102347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cb56b-25a6-49b6-b7de-59b93e307a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:13.102618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cb56b-25a6-49b6-b7de-59b93e307a2c None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:13.103596 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cb56b-25a6-49b6-b7de-59b93e307a2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:13.103900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cb56b-25a6-49b6-b7de-59b93e307a2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:13.105464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cb56b-25a6-49b6-b7de-59b93e307a2c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:13.125039 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 478/946] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 4 05:42:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:13.130424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cb56b-25a6-49b6-b7de-59b93e307a2c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:13.182096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60093180-1486-4e34-93b9-ade4814a4a76 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:13.229008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efb8c0-df55-4fdb-80cd-cbb8f909f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:13.230592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efb8c0-df55-4fdb-80cd-cbb8f909f8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:13.230592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efb8c0-df55-4fdb-80cd-cbb8f909f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:13.230592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efb8c0-df55-4fdb-80cd-cbb8f909f8c9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:13.231041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efb8c0-df55-4fdb-80cd-cbb8f909f8c9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:13.232488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efb8c0-df55-4fdb-80cd-cbb8f909f8c9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-c9efb8c0-df55-4fdb-80cd-cbb8f909f8c9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:13.245427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:13.246728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:13.248013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 469/947] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:13.257354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efb8c0-df55-4fdb-80cd-cbb8f909f8c9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:13.257556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cb56b-25a6-49b6-b7de-59b93e307a2c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:13.292646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2b66a-a245-4f60-9a54-9cbb2ca68a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:13.293552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2b66a-a245-4f60-9a54-9cbb2ca68a57 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:13.293962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2b66a-a245-4f60-9a54-9cbb2ca68a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:13.294251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2b66a-a245-4f60-9a54-9cbb2ca68a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:13.304982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2b66a-a245-4f60-9a54-9cbb2ca68a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:13.306639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2b66a-a245-4f60-9a54-9cbb2ca68a57 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-17c2b66a-a245-4f60-9a54-9cbb2ca68a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:13.326265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:13.327403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:13.328562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:13.328562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:13.328562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:13.328562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:13.329619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 479/948] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:13.335241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2b66a-a245-4f60-9a54-9cbb2ca68a57 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:13.336810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:13.359885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:13.361989 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efb8c0-df55-4fdb-80cd-cbb8f909f8c9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:13.402829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084aec8-6703-4818-8ccd-4213d9f67509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:13.403544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084aec8-6703-4818-8ccd-4213d9f67509 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:13.403841 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084aec8-6703-4818-8ccd-4213d9f67509 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:13.404099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084aec8-6703-4818-8ccd-4213d9f67509 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:13.404809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084aec8-6703-4818-8ccd-4213d9f67509 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:13.405109 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084aec8-6703-4818-8ccd-4213d9f67509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:13.406645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084aec8-6703-4818-8ccd-4213d9f67509 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:13.411900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 470/949] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 4 05:42:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:13.417379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084aec8-6703-4818-8ccd-4213d9f67509 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:13.468054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2b66a-a245-4f60-9a54-9cbb2ca68a57 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:13.515394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c0033-c12a-43e9-9e0f-1b54ebd5b230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:13.516178 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c0033-c12a-43e9-9e0f-1b54ebd5b230 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:13.516530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c0033-c12a-43e9-9e0f-1b54ebd5b230 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:13.516875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c0033-c12a-43e9-9e0f-1b54ebd5b230 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:13.517671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c0033-c12a-43e9-9e0f-1b54ebd5b230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:13.518043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c0033-c12a-43e9-9e0f-1b54ebd5b230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:13.519651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c0033-c12a-43e9-9e0f-1b54ebd5b230 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:13.532788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 480/950] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:13.537407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084aec8-6703-4818-8ccd-4213d9f67509 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:13.538426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c0033-c12a-43e9-9e0f-1b54ebd5b230 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:13.579007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29380473-8730-4ea1-815c-a2a05db0ab7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:13.580016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29380473-8730-4ea1-815c-a2a05db0ab7f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:13.580458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29380473-8730-4ea1-815c-a2a05db0ab7f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:13.580918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29380473-8730-4ea1-815c-a2a05db0ab7f None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:13.581761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29380473-8730-4ea1-815c-a2a05db0ab7f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:13.582190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29380473-8730-4ea1-815c-a2a05db0ab7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:13.583774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29380473-8730-4ea1-815c-a2a05db0ab7f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:13.589421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 471/951] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 4 05:42:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:13.597409 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29380473-8730-4ea1-815c-a2a05db0ab7f None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:13.601533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:13.636355 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:13.666574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c0033-c12a-43e9-9e0f-1b54ebd5b230 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:13.707064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53b1c4-022a-4326-ad85-06f83a21a284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:13.708078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53b1c4-022a-4326-ad85-06f83a21a284 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:13.708423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53b1c4-022a-4326-ad85-06f83a21a284 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:13.708762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53b1c4-022a-4326-ad85-06f83a21a284 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:13.716813 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53b1c4-022a-4326-ad85-06f83a21a284 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:13.718513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53b1c4-022a-4326-ad85-06f83a21a284 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:13.726575 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29380473-8730-4ea1-815c-a2a05db0ab7f None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-9d53b1c4-022a-4326-ad85-06f83a21a284 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:42:13.734131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:13.735050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:13.735981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:42:13.735981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:42:13.735981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:42:13.735981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:42:13.735981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:42:13.735981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:42:13.735981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:13.735981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:13.735981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 481/952] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:13.741141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53b1c4-022a-4326-ad85-06f83a21a284 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:13.770420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e07d-8287-41ba-8129-b251432edde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:13.772113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e07d-8287-41ba-8129-b251432edde9 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:13.772883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e07d-8287-41ba-8129-b251432edde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:13.773402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e07d-8287-41ba-8129-b251432edde9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:13.775098 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935e07d-8287-41ba-8129-b251432edde9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:13.777300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935e07d-8287-41ba-8129-b251432edde9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:13.805229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 472/953] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 4 05:42:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:13.866779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53b1c4-022a-4326-ad85-06f83a21a284 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:13.910403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:13.916115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d23ac8-e165-494a-a4ae-20162d79a397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:13.916912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d23ac8-e165-494a-a4ae-20162d79a397 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:13.917037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d23ac8-e165-494a-a4ae-20162d79a397 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:13.917298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d23ac8-e165-494a-a4ae-20162d79a397 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:13.918109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d23ac8-e165-494a-a4ae-20162d79a397 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:13.918419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d23ac8-e165-494a-a4ae-20162d79a397 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:13.920266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d23ac8-e165-494a-a4ae-20162d79a397 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:13.939216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 482/954] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:42:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:13.940941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e07d-8287-41ba-8129-b251432edde9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:14.017171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:14.061184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e07d-8287-41ba-8129-b251432edde9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:14.096209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfe456-fd08-42c6-901d-5a7fd1571185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:14.097122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfe456-fd08-42c6-901d-5a7fd1571185 None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:14.097403 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfe456-fd08-42c6-901d-5a7fd1571185 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:14.097669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfe456-fd08-42c6-901d-5a7fd1571185 None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/884ed6ddb71647088b33dc9623143b9c/roles/85e17d8dc6cb480abe4e2bb231995a79` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:14.099910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbfe456-fd08-42c6-901d-5a7fd1571185 None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=884ed6ddb71647088b33dc9623143b9c, role_id=85e17d8dc6cb480abe4e2bb231995a79)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:14.101350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbfe456-fd08-42c6-901d-5a7fd1571185 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:14.118533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 473/955] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:42:13 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/884ed6ddb71647088b33dc9623143b9c/roles/85e17d8dc6cb480abe4e2bb231995a79 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:42:14.181595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f00b1-1f9e-4442-8113-1f2626d6760a None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:14.181806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f00b1-1f9e-4442-8113-1f2626d6760a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:14.182072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f00b1-1f9e-4442-8113-1f2626d6760a None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:14.183608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 483/956] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:14.189264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9605f-3145-486e-9410-108044042298 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:14.189814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9605f-3145-486e-9410-108044042298 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:14.190245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9605f-3145-486e-9410-108044042298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:14.209420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-43b9605f-3145-486e-9410-108044042298 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:14.210970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-43b9605f-3145-486e-9410-108044042298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:14.292132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 474/957] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:14.316906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f00b1-1f9e-4442-8113-1f2626d6760a None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:14.419924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f00b1-1f9e-4442-8113-1f2626d6760a None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:14.453683 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35170b-6352-4b83-abe8-4e60c95b7608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:14.454707 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35170b-6352-4b83-abe8-4e60c95b7608 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:14.455135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35170b-6352-4b83-abe8-4e60c95b7608 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:14.455601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35170b-6352-4b83-abe8-4e60c95b7608 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:14.456440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35170b-6352-4b83-abe8-4e60c95b7608 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:14.458083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35170b-6352-4b83-abe8-4e60c95b7608 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-bc35170b-6352-4b83-abe8-4e60c95b7608 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:14.470173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:14.471549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 05:42:14.472627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:14.475171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 484/958] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:14.483319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9605f-3145-486e-9410-108044042298 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:14.544306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:14.595514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9605f-3145-486e-9410-108044042298 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:14.630181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8af10f-a1d0-40ba-8e6b-ef500b0c8a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:14.630865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8af10f-a1d0-40ba-8e6b-ef500b0c8a9c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:14.631143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8af10f-a1d0-40ba-8e6b-ef500b0c8a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:14.631442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8af10f-a1d0-40ba-8e6b-ef500b0c8a9c None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:14.632156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8af10f-a1d0-40ba-8e6b-ef500b0c8a9c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:14.632453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8af10f-a1d0-40ba-8e6b-ef500b0c8a9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:14.633821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8af10f-a1d0-40ba-8e6b-ef500b0c8a9c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:14.639404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 475/959] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:42:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:14.644751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35170b-6352-4b83-abe8-4e60c95b7608 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:14.707075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:14.752871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35170b-6352-4b83-abe8-4e60c95b7608 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:14.786728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838fc3f-85d8-490c-87c6-a55e16be0c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:14.787488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838fc3f-85d8-490c-87c6-a55e16be0c21 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:14.787769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838fc3f-85d8-490c-87c6-a55e16be0c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:14.788074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838fc3f-85d8-490c-87c6-a55e16be0c21 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:14.788782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838fc3f-85d8-490c-87c6-a55e16be0c21 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:14.789086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838fc3f-85d8-490c-87c6-a55e16be0c21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:14.790516 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838fc3f-85d8-490c-87c6-a55e16be0c21 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:14.795367 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 485/960] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:42:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:15.224750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18745-af50-46e7-9b34-519a4853fbd7 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:15.225169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18745-af50-46e7-9b34-519a4853fbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:15.225341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18745-af50-46e7-9b34-519a4853fbd7 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:15.226946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 476/961] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:15.232426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f04237-b422-45c0-a08f-c7f4512e8a1d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:15.232719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f04237-b422-45c0-a08f-c7f4512e8a1d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:15.232998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f04237-b422-45c0-a08f-c7f4512e8a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:15.250540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-88f04237-b422-45c0-a08f-c7f4512e8a1d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:15.251789 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-88f04237-b422-45c0-a08f-c7f4512e8a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:15.322374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 486/962] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:15.346933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c18745-af50-46e7-9b34-519a4853fbd7 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:15.449968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c18745-af50-46e7-9b34-519a4853fbd7 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:15.464608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:15.484689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c3e81-cccb-4238-8e09-3ae38b428cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:15.485435 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c3e81-cccb-4238-8e09-3ae38b428cf8 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:15.485639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c3e81-cccb-4238-8e09-3ae38b428cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:15.485908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c3e81-cccb-4238-8e09-3ae38b428cf8 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:15.486607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c3e81-cccb-4238-8e09-3ae38b428cf8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:15.488116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c3e81-cccb-4238-8e09-3ae38b428cf8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:15.507695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.sql.core [None req-896c3e81-cccb-4238-8e09-3ae38b428cf8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 05:42:15.507695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 05:42:15.507695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [parameters: {'user_id': 'c9cd331b5ea24fc0ae02cb2f75b740eb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 05:42:15.507695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80129) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-896c3e81-cccb-4238-8e09-3ae38b428cf8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application conn.query(q) Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application result.read() Jul 04 05:42:15.519306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 05:42:15.520391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application next(self.gen) Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application yield resource Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application next(self.gen) Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise self.value Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application session.commit() Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 05:42:15.521511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self.session.flush() Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application compat.raise_( Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 05:42:15.522659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 05:42:15.523907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application conn.query(q) Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application result.read() Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 05:42:15.524999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c9cd331b5ea24fc0ae02cb2f75b740eb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:15.526179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 04 05:42:15.527171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 04 05:42:15.528291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 04 05:42:15.528291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 04 05:42:15.528291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 05:42:15.528291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 05:42:15.528291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 05:42:15.528291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:15.528291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 477/963] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jul 4 05:42:15 2022] POST /identity/v3/users => generated 160 bytes in 170 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:15.528681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f04237-b422-45c0-a08f-c7f4512e8a1d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:15.587655 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:15.639457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f04237-b422-45c0-a08f-c7f4512e8a1d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:15.674913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a077d-2417-42e5-839b-bc4c410c948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:15.676525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a077d-2417-42e5-839b-bc4c410c948b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:15.676980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a077d-2417-42e5-839b-bc4c410c948b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:15.677422 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a077d-2417-42e5-839b-bc4c410c948b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:15.685446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a077d-2417-42e5-839b-bc4c410c948b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:15.687118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a077d-2417-42e5-839b-bc4c410c948b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-093a077d-2417-42e5-839b-bc4c410c948b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:42:15.702651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:15.703640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:15.704615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:15.704615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:15.704615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:42:15.704615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:42:15.704615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:42:15.704615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:42:15.704615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:42:15.704615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:42:15.704615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 05:42:15.704615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:15.706867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 487/964] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:42:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:15.712734 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c3e81-cccb-4238-8e09-3ae38b428cf8 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:15.820812 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c3e81-cccb-4238-8e09-3ae38b428cf8 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:15.854923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d685a5d-1988-493f-ad81-d20a4e4ecd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:15.855642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d685a5d-1988-493f-ad81-d20a4e4ecd2a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:15.855920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d685a5d-1988-493f-ad81-d20a4e4ecd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:15.856189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d685a5d-1988-493f-ad81-d20a4e4ecd2a None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:15.857003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d685a5d-1988-493f-ad81-d20a4e4ecd2a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:15.857304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d685a5d-1988-493f-ad81-d20a4e4ecd2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:15.858950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d685a5d-1988-493f-ad81-d20a4e4ecd2a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:15.898154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 478/965] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 4 05:42:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:15.953129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26aae41-7fcf-4c7e-9d84-aec47b88b6d0 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:15.953432 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26aae41-7fcf-4c7e-9d84-aec47b88b6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:15.953683 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26aae41-7fcf-4c7e-9d84-aec47b88b6d0 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:15.955336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 488/966] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:15.961559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84b7ad-709d-44bb-a103-a774cad4babb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:15.961843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84b7ad-709d-44bb-a103-a774cad4babb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:15.962205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84b7ad-709d-44bb-a103-a774cad4babb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:15.980948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-8f84b7ad-709d-44bb-a103-a774cad4babb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:15.982509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8f84b7ad-709d-44bb-a103-a774cad4babb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:16.011760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:16.063387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 479/967] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:16.081430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26aae41-7fcf-4c7e-9d84-aec47b88b6d0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:16.184739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26aae41-7fcf-4c7e-9d84-aec47b88b6d0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:16.196380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:16.220891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1a1e4-5dd2-4653-addb-3627def83fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:16.221702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1a1e4-5dd2-4653-addb-3627def83fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.221975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1a1e4-5dd2-4653-addb-3627def83fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.222239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1a1e4-5dd2-4653-addb-3627def83fc0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.222879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1a1e4-5dd2-4653-addb-3627def83fc0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:16.224277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1a1e4-5dd2-4653-addb-3627def83fc0 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-92a1a1e4-5dd2-4653-addb-3627def83fc0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:16.236110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:16.237103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:16.237958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:16.239305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 489/968] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:16.243726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84b7ad-709d-44bb-a103-a774cad4babb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:16.286798 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d5a20-3038-4412-9572-11ba4664b149 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.287371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d5a20-3038-4412-9572-11ba4664b149 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.287649 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d5a20-3038-4412-9572-11ba4664b149 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.289154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 490/969] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:42:16.294029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495658fc-3364-49c5-b31f-2f6b59709e92 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.294311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495658fc-3364-49c5-b31f-2f6b59709e92 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.294582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495658fc-3364-49c5-b31f-2f6b59709e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.311478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-495658fc-3364-49c5-b31f-2f6b59709e92 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:16.312667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-495658fc-3364-49c5-b31f-2f6b59709e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:16.352378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84b7ad-709d-44bb-a103-a774cad4babb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:16.386553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 491/970] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:16.387404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813371e-3fcc-40bf-9bf8-c1e206c52fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:16.388080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813371e-3fcc-40bf-9bf8-c1e206c52fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.388335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813371e-3fcc-40bf-9bf8-c1e206c52fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.388621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813371e-3fcc-40bf-9bf8-c1e206c52fb6 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.389354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813371e-3fcc-40bf-9bf8-c1e206c52fb6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:16.389649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813371e-3fcc-40bf-9bf8-c1e206c52fb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:16.391013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813371e-3fcc-40bf-9bf8-c1e206c52fb6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:16.396514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 480/971] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 4 05:42:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:16.402253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495658fc-3364-49c5-b31f-2f6b59709e92 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:16.409906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813371e-3fcc-40bf-9bf8-c1e206c52fb6 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:16.437886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:16.512700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495658fc-3364-49c5-b31f-2f6b59709e92 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:16.515164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:16.519860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813371e-3fcc-40bf-9bf8-c1e206c52fb6 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:16.561308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fd0ac-5e2e-4ab9-a594-0bbc468f6301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:16.561989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fd0ac-5e2e-4ab9-a594-0bbc468f6301 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.562269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fd0ac-5e2e-4ab9-a594-0bbc468f6301 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.562533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fd0ac-5e2e-4ab9-a594-0bbc468f6301 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.563269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923fd0ac-5e2e-4ab9-a594-0bbc468f6301 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:16.563585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923fd0ac-5e2e-4ab9-a594-0bbc468f6301 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:16.564977 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923fd0ac-5e2e-4ab9-a594-0bbc468f6301 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:16.569605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81953c7f-a5b4-411d-83df-967bff4e2114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:16.570926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81953c7f-a5b4-411d-83df-967bff4e2114 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.571432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81953c7f-a5b4-411d-83df-967bff4e2114 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.571925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81953c7f-a5b4-411d-83df-967bff4e2114 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.577302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 492/972] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 4 05:42:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:16.578096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fd0ac-5e2e-4ab9-a594-0bbc468f6301 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:16.585369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81953c7f-a5b4-411d-83df-967bff4e2114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-81953c7f-a5b4-411d-83df-967bff4e2114 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:16.594782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:16.595949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:16.596839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 481/973] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 4 05:42:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:16.603875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81953c7f-a5b4-411d-83df-967bff4e2114 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:16.689975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fd0ac-5e2e-4ab9-a594-0bbc468f6301 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:16.696062 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:16.711569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81953c7f-a5b4-411d-83df-967bff4e2114 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:16.725969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343872-46a5-4895-8381-e34c0765750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:16.726625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343872-46a5-4895-8381-e34c0765750c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.726905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343872-46a5-4895-8381-e34c0765750c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.727177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343872-46a5-4895-8381-e34c0765750c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.728068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60343872-46a5-4895-8381-e34c0765750c None admin] RBAC: Adding query filter params (interface=admin, service_id=94636d724a8a41a6b90b664210e01442, region_id=RegionOne) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:16.728380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60343872-46a5-4895-8381-e34c0765750c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:16.729772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60343872-46a5-4895-8381-e34c0765750c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:16.734932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 493/974] 10.222.0.42 () {56 vars in 1249 bytes} [Mon Jul 4 05:42:16 2022] GET /identity/v3/endpoints?service_id=94636d724a8a41a6b90b664210e01442&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:16.740467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343872-46a5-4895-8381-e34c0765750c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:16.753119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27fa4-473e-40c1-8349-916ed3d98a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:16.753792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27fa4-473e-40c1-8349-916ed3d98a9a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.754073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27fa4-473e-40c1-8349-916ed3d98a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.754342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27fa4-473e-40c1-8349-916ed3d98a9a None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.755202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27fa4-473e-40c1-8349-916ed3d98a9a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:16.755535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27fa4-473e-40c1-8349-916ed3d98a9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:16.756991 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27fa4-473e-40c1-8349-916ed3d98a9a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:16.766320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 482/975] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 4 05:42:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:16.771799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27fa4-473e-40c1-8349-916ed3d98a9a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:16.777233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:16.849598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343872-46a5-4895-8381-e34c0765750c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:16.875626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27fa4-473e-40c1-8349-916ed3d98a9a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:16.883017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13e81a-52d1-4484-b0ba-fbef625fba78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:16.883715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13e81a-52d1-4484-b0ba-fbef625fba78 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.884018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13e81a-52d1-4484-b0ba-fbef625fba78 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.884289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13e81a-52d1-4484-b0ba-fbef625fba78 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.884963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13e81a-52d1-4484-b0ba-fbef625fba78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:16.886346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13e81a-52d1-4484-b0ba-fbef625fba78 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:16.892376 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 494/976] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 4 05:42:16 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:42:16.893482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e725e55-e0b5-46bb-b0cf-30eee131b822 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.893768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e725e55-e0b5-46bb-b0cf-30eee131b822 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.894043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e725e55-e0b5-46bb-b0cf-30eee131b822 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.895273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 495/977] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:16.901002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dbc80-26ed-46f8-aca1-d9f1825b9d15 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.901324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dbc80-26ed-46f8-aca1-d9f1825b9d15 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.901554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dbc80-26ed-46f8-aca1-d9f1825b9d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.919446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-f79dbc80-26ed-46f8-aca1-d9f1825b9d15 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:16.920692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f79dbc80-26ed-46f8-aca1-d9f1825b9d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:16.925980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981e11-9b42-4119-8cb9-008e58259337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:16.927166 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981e11-9b42-4119-8cb9-008e58259337 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:16.927619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981e11-9b42-4119-8cb9-008e58259337 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:16.928057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981e11-9b42-4119-8cb9-008e58259337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:16.941079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c981e11-9b42-4119-8cb9-008e58259337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:16.942592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c981e11-9b42-4119-8cb9-008e58259337 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-5c981e11-9b42-4119-8cb9-008e58259337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:16.961342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:16.962486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:16.963564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:16.963564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:16.963564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:16.963564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:16.963564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 483/978] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:16.970131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981e11-9b42-4119-8cb9-008e58259337 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:16.973105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:17.001582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 496/979] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:17.024568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dbc80-26ed-46f8-aca1-d9f1825b9d15 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:17.055218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:17.083440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981e11-9b42-4119-8cb9-008e58259337 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:17.119602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc5f32-7f35-4829-9f6e-3e89e2a7703f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:17.120452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc5f32-7f35-4829-9f6e-3e89e2a7703f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:17.120739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc5f32-7f35-4829-9f6e-3e89e2a7703f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:17.121001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc5f32-7f35-4829-9f6e-3e89e2a7703f None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:17.121719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc5f32-7f35-4829-9f6e-3e89e2a7703f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:17.122019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc5f32-7f35-4829-9f6e-3e89e2a7703f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:17.123633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc5f32-7f35-4829-9f6e-3e89e2a7703f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:17.132885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dbc80-26ed-46f8-aca1-d9f1825b9d15 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:17.141342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 484/980] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:17.142340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc5f32-7f35-4829-9f6e-3e89e2a7703f None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:17.168746 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4491fc-f707-475e-862f-50c627abb7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:17.169417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4491fc-f707-475e-862f-50c627abb7c4 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:17.169695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4491fc-f707-475e-862f-50c627abb7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:17.169966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4491fc-f707-475e-862f-50c627abb7c4 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:17.170685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4491fc-f707-475e-862f-50c627abb7c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:17.172094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4491fc-f707-475e-862f-50c627abb7c4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:17.189119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 497/981] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:42:17 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:42:17.208076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:17.252179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc5f32-7f35-4829-9f6e-3e89e2a7703f None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:17.296396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb570448-56a8-4d78-a67b-af1722c503e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:17.297239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb570448-56a8-4d78-a67b-af1722c503e6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:17.297504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb570448-56a8-4d78-a67b-af1722c503e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:17.297823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb570448-56a8-4d78-a67b-af1722c503e6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:17.306150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb570448-56a8-4d78-a67b-af1722c503e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:17.307673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb570448-56a8-4d78-a67b-af1722c503e6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-fb570448-56a8-4d78-a67b-af1722c503e6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:42:17.324520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:17.325565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:17.331561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 485/982] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 4 05:42:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:17.332179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4491fc-f707-475e-862f-50c627abb7c4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:17.423329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:17.439606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4491fc-f707-475e-862f-50c627abb7c4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:17.475427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bda13e-29cd-455e-b459-ba05a918e040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:17.476112 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bda13e-29cd-455e-b459-ba05a918e040 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:17.476391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bda13e-29cd-455e-b459-ba05a918e040 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:17.476666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bda13e-29cd-455e-b459-ba05a918e040 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:17.477509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bda13e-29cd-455e-b459-ba05a918e040 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:17.477849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bda13e-29cd-455e-b459-ba05a918e040 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:17.479319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bda13e-29cd-455e-b459-ba05a918e040 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:17.497364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 498/983] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 4 05:42:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:17.504275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb570448-56a8-4d78-a67b-af1722c503e6 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:17.610797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb570448-56a8-4d78-a67b-af1722c503e6 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:17.629252 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:17.648041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941773c1-b174-4039-a9e4-ac776494491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:17.648699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941773c1-b174-4039-a9e4-ac776494491a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:17.648978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941773c1-b174-4039-a9e4-ac776494491a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:17.649246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941773c1-b174-4039-a9e4-ac776494491a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:17.722326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941773c1-b174-4039-a9e4-ac776494491a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:17.723734 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941773c1-b174-4039-a9e4-ac776494491a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:17.823759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 486/984] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:17 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 05:42:17.829314 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bda13e-29cd-455e-b459-ba05a918e040 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:17.932385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bda13e-29cd-455e-b459-ba05a918e040 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:17.967994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39b7f7-dbb4-414d-bfa5-f263668e60ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:17.968798 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39b7f7-dbb4-414d-bfa5-f263668e60ea None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:17.969076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39b7f7-dbb4-414d-bfa5-f263668e60ea None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:17.969345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39b7f7-dbb4-414d-bfa5-f263668e60ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:17.979974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39b7f7-dbb4-414d-bfa5-f263668e60ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:17.981325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39b7f7-dbb4-414d-bfa5-f263668e60ea None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:17.984680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-be39b7f7-dbb4-414d-bfa5-f263668e60ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:18.001498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:18.002637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:18.003624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:18.003624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:18.003624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:18.003624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:18.003624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 499/985] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:18.010742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941773c1-b174-4039-a9e4-ac776494491a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:18.031218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:18.113722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941773c1-b174-4039-a9e4-ac776494491a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:18.148197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a69b2-fb17-49bc-9cbd-4ce7dfb8acb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:18.148866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a69b2-fb17-49bc-9cbd-4ce7dfb8acb4 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:18.149147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a69b2-fb17-49bc-9cbd-4ce7dfb8acb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:18.149424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a69b2-fb17-49bc-9cbd-4ce7dfb8acb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:18.150187 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a69b2-fb17-49bc-9cbd-4ce7dfb8acb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:18.150495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a69b2-fb17-49bc-9cbd-4ce7dfb8acb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:18.151906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a69b2-fb17-49bc-9cbd-4ce7dfb8acb4 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:18.163433 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 487/986] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:18.168985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39b7f7-dbb4-414d-bfa5-f263668e60ea None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:18.272119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39b7f7-dbb4-414d-bfa5-f263668e60ea None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:18.307411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ee20e-3237-4d7a-afc8-753b7b6a8554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:18.308228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee20e-3237-4d7a-afc8-753b7b6a8554 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:18.308505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee20e-3237-4d7a-afc8-753b7b6a8554 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:18.308770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee20e-3237-4d7a-afc8-753b7b6a8554 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:18.316729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ee20e-3237-4d7a-afc8-753b7b6a8554 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:18.318188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ee20e-3237-4d7a-afc8-753b7b6a8554 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-174ee20e-3237-4d7a-afc8-753b7b6a8554 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:42:18.332638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:18.333671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:18.334579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:42:18.334579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:42:18.334579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:42:18.334579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:42:18.334579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:42:18.334579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:42:18.334579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:18.334579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:18.334579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 500/987] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:18.339782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a69b2-fb17-49bc-9cbd-4ce7dfb8acb4 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:18.421632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:18.446645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a69b2-fb17-49bc-9cbd-4ce7dfb8acb4 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:18.480425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caec13f-0414-46cf-8a6b-9a5bb4de69f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:18.481100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caec13f-0414-46cf-8a6b-9a5bb4de69f7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:18.481379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caec13f-0414-46cf-8a6b-9a5bb4de69f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:18.481650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caec13f-0414-46cf-8a6b-9a5bb4de69f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:18.482434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caec13f-0414-46cf-8a6b-9a5bb4de69f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:18.482740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caec13f-0414-46cf-8a6b-9a5bb4de69f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:18.484219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caec13f-0414-46cf-8a6b-9a5bb4de69f7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:18.497050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 488/988] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:42:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:18.502927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ee20e-3237-4d7a-afc8-753b7b6a8554 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:18.566256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:18.614909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ee20e-3237-4d7a-afc8-753b7b6a8554 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:18.655128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7bbf6-79c6-4c42-83fd-f1fbcd18b52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:18.656255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7bbf6-79c6-4c42-83fd-f1fbcd18b52b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:18.656754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7bbf6-79c6-4c42-83fd-f1fbcd18b52b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:18.657230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7bbf6-79c6-4c42-83fd-f1fbcd18b52b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:18.658328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7bbf6-79c6-4c42-83fd-f1fbcd18b52b None admin] RBAC: Adding query filter params (role.id=85e17d8dc6cb480abe4e2bb231995a79, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=884ed6ddb71647088b33dc9623143b9c) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:18.658864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7bbf6-79c6-4c42-83fd-f1fbcd18b52b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:18.660575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7bbf6-79c6-4c42-83fd-f1fbcd18b52b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:18.667613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 501/989] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:18 2022] GET /identity/v3/role_assignments?user.id=884ed6ddb71647088b33dc9623143b9c&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=85e17d8dc6cb480abe4e2bb231995a79 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:18.944645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb255924-cd72-413b-99f1-f3bb816584d7 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:18.945164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb255924-cd72-413b-99f1-f3bb816584d7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:18.945164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb255924-cd72-413b-99f1-f3bb816584d7 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:18.946682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 489/990] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:18.951919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c7f7-13b8-4860-bc36-500f9fe793c8 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:18.952365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c7f7-13b8-4860-bc36-500f9fe793c8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:18.952443 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c7f7-13b8-4860-bc36-500f9fe793c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:18.970265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b8e1c7f7-13b8-4860-bc36-500f9fe793c8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:18.971517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b8e1c7f7-13b8-4860-bc36-500f9fe793c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:19.047277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 502/991] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:19.067204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb255924-cd72-413b-99f1-f3bb816584d7 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:19.174424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb255924-cd72-413b-99f1-f3bb816584d7 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:19.213633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28694537-77c8-4b89-9143-8792e4da66dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:19.213633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28694537-77c8-4b89-9143-8792e4da66dd None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:19.213633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28694537-77c8-4b89-9143-8792e4da66dd None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:19.213633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28694537-77c8-4b89-9143-8792e4da66dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:19.216055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28694537-77c8-4b89-9143-8792e4da66dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:19.216055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28694537-77c8-4b89-9143-8792e4da66dd None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:19.218588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5af923-556b-421d-9b0b-c35821d3a6a6 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:19.218877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5af923-556b-421d-9b0b-c35821d3a6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:19.219157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5af923-556b-421d-9b0b-c35821d3a6a6 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:19.220804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 503/992] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:19.225954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aa5ab-7579-4e85-82f5-b8e21c31e3c4 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:19.226231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aa5ab-7579-4e85-82f5-b8e21c31e3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:19.226500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aa5ab-7579-4e85-82f5-b8e21c31e3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-28694537-77c8-4b89-9143-8792e4da66dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:19.236358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:19.237384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:19.240711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 490/993] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 4 05:42:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:19.243460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28694537-77c8-4b89-9143-8792e4da66dd None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:19.245924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-de3aa5ab-7579-4e85-82f5-b8e21c31e3c4 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:19.246618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:19.247274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-de3aa5ab-7579-4e85-82f5-b8e21c31e3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:19.294854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:19.317180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 504/994] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:19.340150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3aa5ab-7579-4e85-82f5-b8e21c31e3c4 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:19.353483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28694537-77c8-4b89-9143-8792e4da66dd None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:19.389094 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259e0e4-3237-4b1a-a47b-21ac978ce121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:19.389859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259e0e4-3237-4b1a-a47b-21ac978ce121 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:19.390140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259e0e4-3237-4b1a-a47b-21ac978ce121 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:19.390426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259e0e4-3237-4b1a-a47b-21ac978ce121 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:19.391524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259e0e4-3237-4b1a-a47b-21ac978ce121 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:19.391524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259e0e4-3237-4b1a-a47b-21ac978ce121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:19.392811 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259e0e4-3237-4b1a-a47b-21ac978ce121 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:19.397720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 491/995] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 4 05:42:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:19.403685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259e0e4-3237-4b1a-a47b-21ac978ce121 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:19.446153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3aa5ab-7579-4e85-82f5-b8e21c31e3c4 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:19.471896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:19.482026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ab67e-71ad-4696-a029-d3cffaab20f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:19.482833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab67e-71ad-4696-a029-d3cffaab20f6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:19.483114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab67e-71ad-4696-a029-d3cffaab20f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:19.483439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab67e-71ad-4696-a029-d3cffaab20f6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:19.484073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ab67e-71ad-4696-a029-d3cffaab20f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:19.485517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ab67e-71ad-4696-a029-d3cffaab20f6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-757ab67e-71ad-4696-a029-d3cffaab20f6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:19.497205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:19.498183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:19.499043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 505/996] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:19.504644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ab67e-71ad-4696-a029-d3cffaab20f6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:19.515078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259e0e4-3237-4b1a-a47b-21ac978ce121 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:19.554341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:19.555530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349379b2-94b3-4b88-a005-98ae9c2c8d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:19.556030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349379b2-94b3-4b88-a005-98ae9c2c8d8d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:19.556595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349379b2-94b3-4b88-a005-98ae9c2c8d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:19.556595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349379b2-94b3-4b88-a005-98ae9c2c8d8d None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:19.557414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349379b2-94b3-4b88-a005-98ae9c2c8d8d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:19.557670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349379b2-94b3-4b88-a005-98ae9c2c8d8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:19.559493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349379b2-94b3-4b88-a005-98ae9c2c8d8d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:19.564716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 492/997] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 4 05:42:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:19.570725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349379b2-94b3-4b88-a005-98ae9c2c8d8d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:19.615315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ab67e-71ad-4696-a029-d3cffaab20f6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:19.650329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc578b-f4e3-4ef9-83da-83e2772934ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:19.651234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc578b-f4e3-4ef9-83da-83e2772934ae None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:19.651542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc578b-f4e3-4ef9-83da-83e2772934ae None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:19.651816 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc578b-f4e3-4ef9-83da-83e2772934ae None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:19.652520 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc578b-f4e3-4ef9-83da-83e2772934ae None admin] RBAC: Adding query filter params (name=compute) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:19.652818 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc578b-f4e3-4ef9-83da-83e2772934ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:19.654196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc578b-f4e3-4ef9-83da-83e2772934ae None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:19.659471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 506/998] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:42:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:19.664912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc578b-f4e3-4ef9-83da-83e2772934ae None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:19.674847 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349379b2-94b3-4b88-a005-98ae9c2c8d8d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:19.704927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:19.712258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c411e-cb8c-482e-8f8a-da01c1ef901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:19.713264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c411e-cb8c-482e-8f8a-da01c1ef901c None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:19.713796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c411e-cb8c-482e-8f8a-da01c1ef901c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:19.714343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c411e-cb8c-482e-8f8a-da01c1ef901c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:19.715408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c411e-cb8c-482e-8f8a-da01c1ef901c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:19.717089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c411e-cb8c-482e-8f8a-da01c1ef901c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:19.734134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 493/999] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 4 05:42:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:19.779589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc578b-f4e3-4ef9-83da-83e2772934ae None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:19.790371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:19.827517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c34a4-3040-4231-ac2a-4e7bb2179107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:19.828193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c34a4-3040-4231-ac2a-4e7bb2179107 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:19.828466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c34a4-3040-4231-ac2a-4e7bb2179107 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:19.828732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c34a4-3040-4231-ac2a-4e7bb2179107 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:19.829443 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c34a4-3040-4231-ac2a-4e7bb2179107 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:19.829740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c34a4-3040-4231-ac2a-4e7bb2179107 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:19.831142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c34a4-3040-4231-ac2a-4e7bb2179107 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:19.836838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 507/1000] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:42:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:19.843030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c411e-cb8c-482e-8f8a-da01c1ef901c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:19.949514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c411e-cb8c-482e-8f8a-da01c1ef901c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:19.983601 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83852a8-ba5d-4354-b722-2449246442d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:19.984335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83852a8-ba5d-4354-b722-2449246442d9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:19.984555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83852a8-ba5d-4354-b722-2449246442d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:19.984828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83852a8-ba5d-4354-b722-2449246442d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:19.985626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83852a8-ba5d-4354-b722-2449246442d9 None admin] RBAC: Adding query filter params (interface=public, service_id=f7ac43ff495f4334acce7a89cd89f7f0, region_id=RegionOne) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:19.985938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83852a8-ba5d-4354-b722-2449246442d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:19.987338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83852a8-ba5d-4354-b722-2449246442d9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:19.992823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 494/1001] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 4 05:42:19 2022] GET /identity/v3/endpoints?service_id=f7ac43ff495f4334acce7a89cd89f7f0&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:19.998649 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c34a4-3040-4231-ac2a-4e7bb2179107 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:20.106294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c34a4-3040-4231-ac2a-4e7bb2179107 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:20.139994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40873e0-96a2-43ad-a1d8-c2adb560129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:20.140668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40873e0-96a2-43ad-a1d8-c2adb560129e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:20.140951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40873e0-96a2-43ad-a1d8-c2adb560129e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:20.141227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40873e0-96a2-43ad-a1d8-c2adb560129e None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:20.141910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40873e0-96a2-43ad-a1d8-c2adb560129e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:20.143331 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40873e0-96a2-43ad-a1d8-c2adb560129e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:20.149877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 508/1002] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 4 05:42:19 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:42:20.733415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808eb82-e9b1-4cd7-9749-7d49391d6885 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:20.733667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808eb82-e9b1-4cd7-9749-7d49391d6885 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:20.733996 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808eb82-e9b1-4cd7-9749-7d49391d6885 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:20.738979 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 495/1003] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:20.740799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fe641-016d-425e-9efd-9d0775916a2e None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:20.741116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fe641-016d-425e-9efd-9d0775916a2e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:20.741367 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fe641-016d-425e-9efd-9d0775916a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:20.758405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-f53fe641-016d-425e-9efd-9d0775916a2e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:20.759693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f53fe641-016d-425e-9efd-9d0775916a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:20.787613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:20.831489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 509/1004] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:20.853752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808eb82-e9b1-4cd7-9749-7d49391d6885 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:20.917033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:20.965859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808eb82-e9b1-4cd7-9749-7d49391d6885 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:20.999401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c070923-18f6-4744-88e9-ee0f889dc5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:21.000197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070923-18f6-4744-88e9-ee0f889dc5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:21.000453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070923-18f6-4744-88e9-ee0f889dc5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:21.000741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070923-18f6-4744-88e9-ee0f889dc5f1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:21.001390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c070923-18f6-4744-88e9-ee0f889dc5f1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:21.002766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c070923-18f6-4744-88e9-ee0f889dc5f1 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-2c070923-18f6-4744-88e9-ee0f889dc5f1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:21.014818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:21.016029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:21.016900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 496/1005] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Jul 4 05:42:20 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:21.024239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fe641-016d-425e-9efd-9d0775916a2e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:21.130141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fe641-016d-425e-9efd-9d0775916a2e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:21.167164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f7d03-8087-469d-8a0e-5713e9b89671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:21.168151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f7d03-8087-469d-8a0e-5713e9b89671 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:21.168639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f7d03-8087-469d-8a0e-5713e9b89671 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:21.169118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f7d03-8087-469d-8a0e-5713e9b89671 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:21.169999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f7d03-8087-469d-8a0e-5713e9b89671 None admin] RBAC: Adding query filter params (name=image) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:21.170482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f7d03-8087-469d-8a0e-5713e9b89671 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:21.172062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f7d03-8087-469d-8a0e-5713e9b89671 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:21.177518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 510/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:21.183118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c070923-18f6-4744-88e9-ee0f889dc5f1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:21.290578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c070923-18f6-4744-88e9-ee0f889dc5f1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:21.308378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:21.329218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05eb07a-2c17-4102-8e09-0cbbb211d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:21.329905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eb07a-2c17-4102-8e09-0cbbb211d18f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:21.330191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eb07a-2c17-4102-8e09-0cbbb211d18f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:21.330428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eb07a-2c17-4102-8e09-0cbbb211d18f None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:21.331162 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05eb07a-2c17-4102-8e09-0cbbb211d18f None admin] RBAC: Adding query filter params (type=image) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:21.331503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05eb07a-2c17-4102-8e09-0cbbb211d18f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:21.332890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05eb07a-2c17-4102-8e09-0cbbb211d18f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:21.343291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 497/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:21.788283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95040e3-44ce-48fb-843d-74689bd9b215 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:21.788475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95040e3-44ce-48fb-843d-74689bd9b215 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:21.788754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95040e3-44ce-48fb-843d-74689bd9b215 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:21.790187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 511/1008] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:21.795569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03aa3c-e214-4aa7-9464-f90ef3687f3e None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:21.795830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03aa3c-e214-4aa7-9464-f90ef3687f3e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:21.796128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03aa3c-e214-4aa7-9464-f90ef3687f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:21.813153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-cf03aa3c-e214-4aa7-9464-f90ef3687f3e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:21.814337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cf03aa3c-e214-4aa7-9464-f90ef3687f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:21.886342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 498/1009] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:21.909381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95040e3-44ce-48fb-843d-74689bd9b215 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:21.939635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:22.020425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95040e3-44ce-48fb-843d-74689bd9b215 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:22.057121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfba55-f9a4-4efa-a8c0-63b78a7b43ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:22.057895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfba55-f9a4-4efa-a8c0-63b78a7b43ce None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:22.058197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfba55-f9a4-4efa-a8c0-63b78a7b43ce None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:22.058475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfba55-f9a4-4efa-a8c0-63b78a7b43ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:22.059699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfba55-f9a4-4efa-a8c0-63b78a7b43ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:22.061156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfba55-f9a4-4efa-a8c0-63b78a7b43ce None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:22.067516 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 512/1010] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:22.069136 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03aa3c-e214-4aa7-9464-f90ef3687f3e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:22.164321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66d617-66bd-4526-b248-94f720c46878 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:22.164724 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66d617-66bd-4526-b248-94f720c46878 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:22.165008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66d617-66bd-4526-b248-94f720c46878 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:22.166496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 513/1011] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:22.171656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493dcf9-a1eb-4a61-a681-289f27b971c7 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:22.171786 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03aa3c-e214-4aa7-9464-f90ef3687f3e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:22.171873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493dcf9-a1eb-4a61-a681-289f27b971c7 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:22.171946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493dcf9-a1eb-4a61-a681-289f27b971c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:22.186339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:22.188514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b493dcf9-a1eb-4a61-a681-289f27b971c7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:22.189750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b493dcf9-a1eb-4a61-a681-289f27b971c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:22.211134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602b3a4-be88-4c77-9c99-8160c6857ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:22.211836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602b3a4-be88-4c77-9c99-8160c6857ba8 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:22.212323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602b3a4-be88-4c77-9c99-8160c6857ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:22.212642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602b3a4-be88-4c77-9c99-8160c6857ba8 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:22.213364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602b3a4-be88-4c77-9c99-8160c6857ba8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:22.214797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602b3a4-be88-4c77-9c99-8160c6857ba8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:22.244353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 499/1012] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:42:22 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:22.267545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 514/1013] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:22.290830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602b3a4-be88-4c77-9c99-8160c6857ba8 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:22.403241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602b3a4-be88-4c77-9c99-8160c6857ba8 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:22.442293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf9fa5-8a04-412b-83bf-b62456e33919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:22.443158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf9fa5-8a04-412b-83bf-b62456e33919 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:22.443486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf9fa5-8a04-412b-83bf-b62456e33919 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:22.444110 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf9fa5-8a04-412b-83bf-b62456e33919 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:22.444804 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf9fa5-8a04-412b-83bf-b62456e33919 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:22.446237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf9fa5-8a04-412b-83bf-b62456e33919 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-87bf9fa5-8a04-412b-83bf-b62456e33919 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:22.459842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:22.460901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:22.461785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 500/1014] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:22.468664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493dcf9-a1eb-4a61-a681-289f27b971c7 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:22.494858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:22.581496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493dcf9-a1eb-4a61-a681-289f27b971c7 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:22.622694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222e7a8-563b-4d38-ad0b-ed055879b90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:22.623394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222e7a8-563b-4d38-ad0b-ed055879b90c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:22.623676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222e7a8-563b-4d38-ad0b-ed055879b90c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:22.623950 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222e7a8-563b-4d38-ad0b-ed055879b90c None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:22.624655 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222e7a8-563b-4d38-ad0b-ed055879b90c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:22.624956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222e7a8-563b-4d38-ad0b-ed055879b90c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:22.626382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222e7a8-563b-4d38-ad0b-ed055879b90c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:22.631516 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 515/1015] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:42:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:22.637140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf9fa5-8a04-412b-83bf-b62456e33919 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:22.667331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:22.740594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf9fa5-8a04-412b-83bf-b62456e33919 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:22.776270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3065952-289d-4e06-b752-57a8d3ba1bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:22.776946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3065952-289d-4e06-b752-57a8d3ba1bad None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:22.777225 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3065952-289d-4e06-b752-57a8d3ba1bad None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:22.777498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3065952-289d-4e06-b752-57a8d3ba1bad None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:22.778210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3065952-289d-4e06-b752-57a8d3ba1bad None admin] RBAC: Adding query filter params (type=compute) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:22.778514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3065952-289d-4e06-b752-57a8d3ba1bad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:22.779922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3065952-289d-4e06-b752-57a8d3ba1bad None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:22.786101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 501/1016] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 4 05:42:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:22.791104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222e7a8-563b-4d38-ad0b-ed055879b90c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:22.884456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:22.896952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222e7a8-563b-4d38-ad0b-ed055879b90c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:22.931358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0f2b5-dc99-4438-90cf-4f85857f25f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:22.931950 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0f2b5-dc99-4438-90cf-4f85857f25f2 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:22.932232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0f2b5-dc99-4438-90cf-4f85857f25f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:22.932508 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0f2b5-dc99-4438-90cf-4f85857f25f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:22.933272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0f2b5-dc99-4438-90cf-4f85857f25f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:22.934665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0f2b5-dc99-4438-90cf-4f85857f25f2 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:22.962498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 516/1017] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 4 05:42:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:42:23.381355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfbb77-8104-4fe4-a3f7-45a6cf46b617 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:23.381888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfbb77-8104-4fe4-a3f7-45a6cf46b617 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:23.381888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfbb77-8104-4fe4-a3f7-45a6cf46b617 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:23.383336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 502/1018] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:23.388896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf283d-7465-4e43-b691-74a00de4a880 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:23.389359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf283d-7465-4e43-b691-74a00de4a880 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:23.389438 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf283d-7465-4e43-b691-74a00de4a880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:23.407034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ffcf283d-7465-4e43-b691-74a00de4a880 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:23.408250 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ffcf283d-7465-4e43-b691-74a00de4a880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:23.485423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 517/1019] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:23 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:23.508267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfbb77-8104-4fe4-a3f7-45a6cf46b617 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:23.605116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:23.616580 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfbb77-8104-4fe4-a3f7-45a6cf46b617 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:23.650069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceac52-d37f-4bbf-be62-cb0899407139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:23.650673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceac52-d37f-4bbf-be62-cb0899407139 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:23.650959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceac52-d37f-4bbf-be62-cb0899407139 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:23.651233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceac52-d37f-4bbf-be62-cb0899407139 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:23.651991 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceac52-d37f-4bbf-be62-cb0899407139 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:23.653457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceac52-d37f-4bbf-be62-cb0899407139 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:23.669605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 503/1020] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:42:23 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:24.355666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15271ce-36a6-4ebf-8ecb-5d86e4cf1226 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:24.356089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15271ce-36a6-4ebf-8ecb-5d86e4cf1226 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:24.356221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15271ce-36a6-4ebf-8ecb-5d86e4cf1226 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:24.357570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 518/1021] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:24.363218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae6460-2813-4eed-94ac-d3ab964ce5b7 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:24.363512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae6460-2813-4eed-94ac-d3ab964ce5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:24.363825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae6460-2813-4eed-94ac-d3ab964ce5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:24.380862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-40ae6460-2813-4eed-94ac-d3ab964ce5b7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:24.382051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-40ae6460-2813-4eed-94ac-d3ab964ce5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:24.466022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 504/1022] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:24.490976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15271ce-36a6-4ebf-8ecb-5d86e4cf1226 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:24.573595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:24.612388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15271ce-36a6-4ebf-8ecb-5d86e4cf1226 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:24.648446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93dc2e0-399f-47b8-815b-af3916f5a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:24.649246 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dc2e0-399f-47b8-815b-af3916f5a920 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:24.649550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dc2e0-399f-47b8-815b-af3916f5a920 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:24.649809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dc2e0-399f-47b8-815b-af3916f5a920 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:24.651089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93dc2e0-399f-47b8-815b-af3916f5a920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:24.652584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93dc2e0-399f-47b8-815b-af3916f5a920 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:24.654385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 519/1023] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:24.660065 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae6460-2813-4eed-94ac-d3ab964ce5b7 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:24.736671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:24.759898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae6460-2813-4eed-94ac-d3ab964ce5b7 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:24.794467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed17c1e-fb29-4cd5-a88b-17743176a373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:24.795156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed17c1e-fb29-4cd5-a88b-17743176a373 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:24.795470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed17c1e-fb29-4cd5-a88b-17743176a373 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:24.795740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed17c1e-fb29-4cd5-a88b-17743176a373 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:24.796473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed17c1e-fb29-4cd5-a88b-17743176a373 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:24.797915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed17c1e-fb29-4cd5-a88b-17743176a373 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:24.819619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 505/1024] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:42:24 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:25.071941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e730d65-1351-4226-9cd6-31d05f6de358 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:25.072348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e730d65-1351-4226-9cd6-31d05f6de358 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:25.072475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e730d65-1351-4226-9cd6-31d05f6de358 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:25.073927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 520/1025] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:25.079671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d15f8-d140-4250-871d-b996c2439567 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:25.079968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d15f8-d140-4250-871d-b996c2439567 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:25.080243 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d15f8-d140-4250-871d-b996c2439567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:25.098126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-6a9d15f8-d140-4250-871d-b996c2439567 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:25.099359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6a9d15f8-d140-4250-871d-b996c2439567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:25.172108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 506/1026] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:25.196411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e730d65-1351-4226-9cd6-31d05f6de358 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:25.304469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e730d65-1351-4226-9cd6-31d05f6de358 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:25.340318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fc88e-177e-41e1-8e04-ac463393afb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:25.341059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fc88e-177e-41e1-8e04-ac463393afb6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:25.341339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fc88e-177e-41e1-8e04-ac463393afb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:25.341614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fc88e-177e-41e1-8e04-ac463393afb6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:25.348904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:25.353421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0fc88e-177e-41e1-8e04-ac463393afb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-8b0fc88e-177e-41e1-8e04-ac463393afb6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:25.362599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:25.363758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:25.364605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:25.365786 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 521/1027] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 4 05:42:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:25.370339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d15f8-d140-4250-871d-b996c2439567 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:25.434208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:25.473256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d15f8-d140-4250-871d-b996c2439567 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:25.507660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6fbdd-970c-48b4-97d8-01dd4a54d608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:25.508703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6fbdd-970c-48b4-97d8-01dd4a54d608 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:25.509305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6fbdd-970c-48b4-97d8-01dd4a54d608 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:25.509785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6fbdd-970c-48b4-97d8-01dd4a54d608 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:25.510764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6fbdd-970c-48b4-97d8-01dd4a54d608 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:25.511290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6fbdd-970c-48b4-97d8-01dd4a54d608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:25.512885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6fbdd-970c-48b4-97d8-01dd4a54d608 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:25.526995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 507/1028] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 4 05:42:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:25.528675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fc88e-177e-41e1-8e04-ac463393afb6 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:25.636629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fc88e-177e-41e1-8e04-ac463393afb6 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:25.672791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e06f70-7207-440e-a5d7-dda7e07dc74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:25.673598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e06f70-7207-440e-a5d7-dda7e07dc74f None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:25.673883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e06f70-7207-440e-a5d7-dda7e07dc74f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:25.674111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e06f70-7207-440e-a5d7-dda7e07dc74f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:25.685406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e06f70-7207-440e-a5d7-dda7e07dc74f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:25.686869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e06f70-7207-440e-a5d7-dda7e07dc74f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-d4e06f70-7207-440e-a5d7-dda7e07dc74f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:25.704604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:25.705517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:25.711109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:25.711109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:25.711109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:25.711109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:25.711109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 522/1029] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:25.713388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6fbdd-970c-48b4-97d8-01dd4a54d608 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:25.763828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5994d-9bfd-4e0f-b889-dade8f7123d1 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:25.764183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5994d-9bfd-4e0f-b889-dade8f7123d1 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:25.764366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5994d-9bfd-4e0f-b889-dade8f7123d1 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:25.765909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 523/1030] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:42:25.771083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbe859-c672-4b56-ac06-53048ceeee7b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:25.771643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbe859-c672-4b56-ac06-53048ceeee7b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:25.771832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbe859-c672-4b56-ac06-53048ceeee7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:25.789661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-9cbbe859-c672-4b56-ac06-53048ceeee7b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:25.790881 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9cbbe859-c672-4b56-ac06-53048ceeee7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:25.829403 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6fbdd-970c-48b4-97d8-01dd4a54d608 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:25.840093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:25.858182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:25.868431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fa926-af27-4997-9614-b4efa795c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:25.869112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fa926-af27-4997-9614-b4efa795c33c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:25.869458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fa926-af27-4997-9614-b4efa795c33c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:25.869748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fa926-af27-4997-9614-b4efa795c33c None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:25.870567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fa926-af27-4997-9614-b4efa795c33c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:25.870878 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fa926-af27-4997-9614-b4efa795c33c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:25.871574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 524/1031] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:25.872295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fa926-af27-4997-9614-b4efa795c33c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:25.887352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 508/1032] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:25.888519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbe859-c672-4b56-ac06-53048ceeee7b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:25.890432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fa926-af27-4997-9614-b4efa795c33c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:25.992875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbe859-c672-4b56-ac06-53048ceeee7b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:25.993191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fa926-af27-4997-9614-b4efa795c33c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:26.030032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3e0fa-5502-4bba-88cb-2bdbbccce3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:26.030838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3e0fa-5502-4bba-88cb-2bdbbccce3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.031118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3e0fa-5502-4bba-88cb-2bdbbccce3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.031437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d050037-0883-4255-97d4-7ed23bf8bea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:26.031674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3e0fa-5502-4bba-88cb-2bdbbccce3d8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.032098 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3e0fa-5502-4bba-88cb-2bdbbccce3d8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:26.032258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d050037-0883-4255-97d4-7ed23bf8bea3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.032497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d050037-0883-4255-97d4-7ed23bf8bea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.032767 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d050037-0883-4255-97d4-7ed23bf8bea3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.033504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3e0fa-5502-4bba-88cb-2bdbbccce3d8 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:26.040222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d050037-0883-4255-97d4-7ed23bf8bea3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:26.041699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d050037-0883-4255-97d4-7ed23bf8bea3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-e5a3e0fa-5502-4bba-88cb-2bdbbccce3d8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:26.045255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:26.046252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:26.054244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 525/1033] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Jul 4 05:42:25 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:26.055060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3e0fa-5502-4bba-88cb-2bdbbccce3d8 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-0d050037-0883-4255-97d4-7ed23bf8bea3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:42:26.058901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:26.059974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:26.060930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 509/1034] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 4 05:42:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:26.066086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d050037-0883-4255-97d4-7ed23bf8bea3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:26.105749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:26.117119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:26.165869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3e0fa-5502-4bba-88cb-2bdbbccce3d8 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:26.176831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d050037-0883-4255-97d4-7ed23bf8bea3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:26.205723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aef014-75fd-4dfb-a63d-db18142d9c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:26.206934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aef014-75fd-4dfb-a63d-db18142d9c5d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.207325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aef014-75fd-4dfb-a63d-db18142d9c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.207621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aef014-75fd-4dfb-a63d-db18142d9c5d None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.208326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aef014-75fd-4dfb-a63d-db18142d9c5d None admin] RBAC: Adding query filter params (name=image) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:26.208631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aef014-75fd-4dfb-a63d-db18142d9c5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:26.210275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aef014-75fd-4dfb-a63d-db18142d9c5d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:26.211363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb177869-c1ec-431a-aa34-26435df9105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:26.212159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb177869-c1ec-431a-aa34-26435df9105c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.212519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb177869-c1ec-431a-aa34-26435df9105c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.212832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb177869-c1ec-431a-aa34-26435df9105c None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.213786 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb177869-c1ec-431a-aa34-26435df9105c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:26.214099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb177869-c1ec-431a-aa34-26435df9105c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:26.215731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb177869-c1ec-431a-aa34-26435df9105c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:26.217354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 526/1035] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:26.222658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aef014-75fd-4dfb-a63d-db18142d9c5d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:26.233106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 510/1036] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 4 05:42:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:26.239066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb177869-c1ec-431a-aa34-26435df9105c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:26.330152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aef014-75fd-4dfb-a63d-db18142d9c5d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:26.339823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:26.341298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb177869-c1ec-431a-aa34-26435df9105c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:26.343789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:26.368762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8cfbd-bb92-44c7-9adc-35ea0f505c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:26.369436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8cfbd-bb92-44c7-9adc-35ea0f505c4e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.369713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8cfbd-bb92-44c7-9adc-35ea0f505c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.369980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8cfbd-bb92-44c7-9adc-35ea0f505c4e None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.370686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8cfbd-bb92-44c7-9adc-35ea0f505c4e None admin] RBAC: Adding query filter params (type=image) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:26.370988 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8cfbd-bb92-44c7-9adc-35ea0f505c4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:26.372416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8cfbd-bb92-44c7-9adc-35ea0f505c4e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:26.377243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 527/1037] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:26.379095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13425812-479b-4b92-9fc9-208301679a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:26.379952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13425812-479b-4b92-9fc9-208301679a54 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.380404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13425812-479b-4b92-9fc9-208301679a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.380822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13425812-479b-4b92-9fc9-208301679a54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.382931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8cfbd-bb92-44c7-9adc-35ea0f505c4e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:26.450763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13425812-479b-4b92-9fc9-208301679a54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:26.452187 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13425812-479b-4b92-9fc9-208301679a54 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:26.490235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8cfbd-bb92-44c7-9adc-35ea0f505c4e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:26.533992 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f8992-600a-4aad-a907-1852c17e8200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:26.534645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f8992-600a-4aad-a907-1852c17e8200 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.534918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f8992-600a-4aad-a907-1852c17e8200 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.535186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f8992-600a-4aad-a907-1852c17e8200 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.536006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f8992-600a-4aad-a907-1852c17e8200 None admin] RBAC: Adding query filter params (interface=public, service_id=d1f6ba2b1dd04647adcd502e29c27087, region_id=RegionOne) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:26.536311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f8992-600a-4aad-a907-1852c17e8200 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:26.537692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f8992-600a-4aad-a907-1852c17e8200 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:26.543213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 528/1038] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 4 05:42:26 2022] GET /identity/v3/endpoints?service_id=d1f6ba2b1dd04647adcd502e29c27087&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:26.548609 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f8992-600a-4aad-a907-1852c17e8200 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:26.563326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 511/1039] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 05:42:26.569213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13425812-479b-4b92-9fc9-208301679a54 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:26.572777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:26.591499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:26.662811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f8992-600a-4aad-a907-1852c17e8200 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:26.684547 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13425812-479b-4b92-9fc9-208301679a54 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:26.697672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b6fbf-d291-4666-a324-f8493f0b1606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:26.698348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b6fbf-d291-4666-a324-f8493f0b1606 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.698629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b6fbf-d291-4666-a324-f8493f0b1606 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.698907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b6fbf-d291-4666-a324-f8493f0b1606 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.699629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b6fbf-d291-4666-a324-f8493f0b1606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:26.701060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b6fbf-d291-4666-a324-f8493f0b1606 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:26.708718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 529/1040] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 4 05:42:26 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:42:26.720998 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dccc8e-8902-49ff-8262-c39e2cd15581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:26.721832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dccc8e-8902-49ff-8262-c39e2cd15581 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.722115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dccc8e-8902-49ff-8262-c39e2cd15581 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.722389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dccc8e-8902-49ff-8262-c39e2cd15581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.734336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dccc8e-8902-49ff-8262-c39e2cd15581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:26.735869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dccc8e-8902-49ff-8262-c39e2cd15581 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-33dccc8e-8902-49ff-8262-c39e2cd15581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:26.755610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:26.756604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:26.757530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:26.757530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:26.757530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:26.757530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:26.757530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 512/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:26.763026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b6fbf-d291-4666-a324-f8493f0b1606 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:26.872307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:26.875385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b6fbf-d291-4666-a324-f8493f0b1606 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:26.910084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6ac0c-0a03-457c-be58-4b7af4598169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:26.910758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6ac0c-0a03-457c-be58-4b7af4598169 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.911035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6ac0c-0a03-457c-be58-4b7af4598169 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.911342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6ac0c-0a03-457c-be58-4b7af4598169 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.912055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6ac0c-0a03-457c-be58-4b7af4598169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:26.912358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6ac0c-0a03-457c-be58-4b7af4598169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:26.913749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6ac0c-0a03-457c-be58-4b7af4598169 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:26.920806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3376b9-b613-4a18-a207-62c97e4a9add None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.921087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3376b9-b613-4a18-a207-62c97e4a9add None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.921361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3376b9-b613-4a18-a207-62c97e4a9add None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.922938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 513/1042] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:42:26.930894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 530/1043] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:26.931044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e0a5b-6719-4737-95ee-d897744737bc None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:26.931044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e0a5b-6719-4737-95ee-d897744737bc None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:26.931044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e0a5b-6719-4737-95ee-d897744737bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:26.932554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6ac0c-0a03-457c-be58-4b7af4598169 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:26.945587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-f63e0a5b-6719-4737-95ee-d897744737bc None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:26.946812 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f63e0a5b-6719-4737-95ee-d897744737bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:27.000709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:27.020493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 514/1044] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:27.035796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6ac0c-0a03-457c-be58-4b7af4598169 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:27.043713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e0a5b-6719-4737-95ee-d897744737bc None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:27.072752 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746354-a9b1-4a3d-8788-56309050319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:27.073909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746354-a9b1-4a3d-8788-56309050319a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:27.074211 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746354-a9b1-4a3d-8788-56309050319a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:27.074495 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746354-a9b1-4a3d-8788-56309050319a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:27.082126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8746354-a9b1-4a3d-8788-56309050319a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:27.083622 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8746354-a9b1-4a3d-8788-56309050319a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-d8746354-a9b1-4a3d-8788-56309050319a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:42:27.097998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:27.098963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:27.105000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:42:27.105000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:42:27.105000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:42:27.105000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:42:27.105000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:42:27.105000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:42:27.105000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:27.105000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:27.105000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 531/1045] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:27.107373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746354-a9b1-4a3d-8788-56309050319a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:27.132090 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:27.156333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e0a5b-6719-4737-95ee-d897744737bc None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:27.191525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ce961-1033-4503-a946-94c8c8fe27ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:27.192341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ce961-1033-4503-a946-94c8c8fe27ac None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:27.192616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ce961-1033-4503-a946-94c8c8fe27ac None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:27.192888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ce961-1033-4503-a946-94c8c8fe27ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:27.203788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ce961-1033-4503-a946-94c8c8fe27ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-602ce961-1033-4503-a946-94c8c8fe27ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:27.211980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:27.212981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:27.213861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:27.214534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 515/1046] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:42:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:27.219051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746354-a9b1-4a3d-8788-56309050319a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:27.219649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ce961-1033-4503-a946-94c8c8fe27ac None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:27.255434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edbfcc-7575-4c94-bbfd-ddf435bb68c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:27.256016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edbfcc-7575-4c94-bbfd-ddf435bb68c4 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:27.256299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edbfcc-7575-4c94-bbfd-ddf435bb68c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:27.256569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edbfcc-7575-4c94-bbfd-ddf435bb68c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:27.257363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edbfcc-7575-4c94-bbfd-ddf435bb68c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:27.257671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edbfcc-7575-4c94-bbfd-ddf435bb68c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:27.259182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edbfcc-7575-4c94-bbfd-ddf435bb68c4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:27.268654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:27.272758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 532/1047] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:42:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:27.279149 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edbfcc-7575-4c94-bbfd-ddf435bb68c4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:27.330546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ce961-1033-4503-a946-94c8c8fe27ac None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:27.363821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:27.370154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baa21f-18fb-4204-8168-01fe934e4dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:27.370826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baa21f-18fb-4204-8168-01fe934e4dac None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:27.371105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baa21f-18fb-4204-8168-01fe934e4dac None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:27.371423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baa21f-18fb-4204-8168-01fe934e4dac None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:27.372172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baa21f-18fb-4204-8168-01fe934e4dac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:27.372476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baa21f-18fb-4204-8168-01fe934e4dac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:27.373854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baa21f-18fb-4204-8168-01fe934e4dac None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:27.382466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 516/1048] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:42:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:27.388124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baa21f-18fb-4204-8168-01fe934e4dac None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:27.390321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edbfcc-7575-4c94-bbfd-ddf435bb68c4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:27.426216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4352a-64f5-4332-812c-ba3c965226ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:27.426876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4352a-64f5-4332-812c-ba3c965226ee None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:27.427152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4352a-64f5-4332-812c-ba3c965226ee None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:27.427451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4352a-64f5-4332-812c-ba3c965226ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:27.428281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4352a-64f5-4332-812c-ba3c965226ee None admin] RBAC: Adding query filter params (role.id=85e17d8dc6cb480abe4e2bb231995a79, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=2cb245e4300e4d5fbdd425e936f4ac77) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:27.428597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4352a-64f5-4332-812c-ba3c965226ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:27.430034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4352a-64f5-4332-812c-ba3c965226ee None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:27.443304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 533/1049] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:27 2022] GET /identity/v3/role_assignments?user.id=2cb245e4300e4d5fbdd425e936f4ac77&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=85e17d8dc6cb480abe4e2bb231995a79 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:27.508386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baa21f-18fb-4204-8168-01fe934e4dac None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:27.527595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:27.554762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a430a-24f8-40f6-bf03-29b484df7e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:27.555779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a430a-24f8-40f6-bf03-29b484df7e64 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:27.556064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a430a-24f8-40f6-bf03-29b484df7e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:27.556355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a430a-24f8-40f6-bf03-29b484df7e64 None admin] PATH_INFO: `/v3/users/4eb9fc26b3bd4c64bd4453df1c8e8438` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:27.573053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a430a-24f8-40f6-bf03-29b484df7e64 None admin] RBAC: Authorizing `identity:get_user(user_id=4eb9fc26b3bd4c64bd4453df1c8e8438)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:27.574541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a430a-24f8-40f6-bf03-29b484df7e64 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:27.576595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 517/1050] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:27 2022] GET /identity/v3/users/4eb9fc26b3bd4c64bd4453df1c8e8438 => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:27.582751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4352a-64f5-4332-812c-ba3c965226ee None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:27.688535 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4352a-64f5-4332-812c-ba3c965226ee None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:27.724536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3747-539b-4246-be35-409c92401dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:27.725194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3747-539b-4246-be35-409c92401dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:27.725519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3747-539b-4246-be35-409c92401dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:27.725799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3747-539b-4246-be35-409c92401dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:27.740876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:27.795866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c3747-539b-4246-be35-409c92401dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:27.797238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c3747-539b-4246-be35-409c92401dc3 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:27.900361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 534/1051] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:42:27.906004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a430a-24f8-40f6-bf03-29b484df7e64 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:28.003787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a430a-24f8-40f6-bf03-29b484df7e64 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:28.036742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e9fe8-1f36-41e4-862a-1d0ec519e836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:28.037605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e9fe8-1f36-41e4-862a-1d0ec519e836 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:28.037881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e9fe8-1f36-41e4-862a-1d0ec519e836 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:28.038153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e9fe8-1f36-41e4-862a-1d0ec519e836 None admin] PATH_INFO: `/v3/projects/a48d4d741809450a82825e526404656c` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:28.039353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e9fe8-1f36-41e4-862a-1d0ec519e836 None admin] RBAC: Authorizing `identity:get_project(project_id=a48d4d741809450a82825e526404656c)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:28.040819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e9fe8-1f36-41e4-862a-1d0ec519e836 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:28.042739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 518/1052] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:42:27 2022] GET /identity/v3/projects/a48d4d741809450a82825e526404656c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:28.048694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3747-539b-4246-be35-409c92401dc3 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:28.095519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:28.153430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3747-539b-4246-be35-409c92401dc3 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:28.188719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e57e2f-56de-4c44-a2f1-c9858cf782eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:28.189373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e57e2f-56de-4c44-a2f1-c9858cf782eb None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:28.189650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e57e2f-56de-4c44-a2f1-c9858cf782eb None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:28.189923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e57e2f-56de-4c44-a2f1-c9858cf782eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:28.190753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e57e2f-56de-4c44-a2f1-c9858cf782eb None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=a48d4d741809450a82825e526404656c, user.id=4eb9fc26b3bd4c64bd4453df1c8e8438) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:28.191059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e57e2f-56de-4c44-a2f1-c9858cf782eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:28.192531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e57e2f-56de-4c44-a2f1-c9858cf782eb None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:28.198644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 535/1053] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:28 2022] GET /identity/v3/role_assignments?user.id=4eb9fc26b3bd4c64bd4453df1c8e8438&scope.project.id=a48d4d741809450a82825e526404656c&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:28.741842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76e44e-1c4d-48a6-80bf-f59f94d0aadc None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:28.742385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76e44e-1c4d-48a6-80bf-f59f94d0aadc None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:28.742385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76e44e-1c4d-48a6-80bf-f59f94d0aadc None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:28.743929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 519/1054] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:28.749329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892b378-887e-4a97-a43c-19dea9f19716 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:28.749734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892b378-887e-4a97-a43c-19dea9f19716 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:28.749902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892b378-887e-4a97-a43c-19dea9f19716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:28.766990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-3892b378-887e-4a97-a43c-19dea9f19716 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:28.768119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3892b378-887e-4a97-a43c-19dea9f19716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:28.835579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 536/1055] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:28.858694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76e44e-1c4d-48a6-80bf-f59f94d0aadc None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:28.897624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:28.968593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76e44e-1c4d-48a6-80bf-f59f94d0aadc None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:29.002956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b4b07-efa0-434d-9384-363e8e6d2909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:29.003801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b4b07-efa0-434d-9384-363e8e6d2909 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:29.004083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b4b07-efa0-434d-9384-363e8e6d2909 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:29.004353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b4b07-efa0-434d-9384-363e8e6d2909 None admin] PATH_INFO: `/v3/services/image` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:29.005007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b4b07-efa0-434d-9384-363e8e6d2909 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:29.006442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b4b07-efa0-434d-9384-363e8e6d2909 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-634b4b07-efa0-434d-9384-363e8e6d2909 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:29.018218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:29.019189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:29.025436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 520/1056] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Jul 4 05:42:28 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:29.026236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892b378-887e-4a97-a43c-19dea9f19716 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:29.066323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:29.129894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892b378-887e-4a97-a43c-19dea9f19716 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:29.166547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817ef73-c60c-4d1e-be7f-ebb986073a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:29.167219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817ef73-c60c-4d1e-be7f-ebb986073a1c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:29.167531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817ef73-c60c-4d1e-be7f-ebb986073a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:29.167806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817ef73-c60c-4d1e-be7f-ebb986073a1c None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:29.168510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817ef73-c60c-4d1e-be7f-ebb986073a1c None admin] RBAC: Adding query filter params (name=image) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:29.168815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817ef73-c60c-4d1e-be7f-ebb986073a1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:29.170203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817ef73-c60c-4d1e-be7f-ebb986073a1c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:29.175586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 537/1057] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:29.181415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b4b07-efa0-434d-9384-363e8e6d2909 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:29.281572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:29.286263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b4b07-efa0-434d-9384-363e8e6d2909 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:29.321132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3ef0c-1b4a-40f4-a822-76b8bce84464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:29.321866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3ef0c-1b4a-40f4-a822-76b8bce84464 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:29.322197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3ef0c-1b4a-40f4-a822-76b8bce84464 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:29.322423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3ef0c-1b4a-40f4-a822-76b8bce84464 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:29.323132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3ef0c-1b4a-40f4-a822-76b8bce84464 None admin] RBAC: Adding query filter params (type=image) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:29.323477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3ef0c-1b4a-40f4-a822-76b8bce84464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:29.324856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3ef0c-1b4a-40f4-a822-76b8bce84464 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:29.329975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 521/1058] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:29.335849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817ef73-c60c-4d1e-be7f-ebb986073a1c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:29.440897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817ef73-c60c-4d1e-be7f-ebb986073a1c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:29.442420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:29.446668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe34823-37c7-477f-9f46-248071234c0c None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:29.446951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe34823-37c7-477f-9f46-248071234c0c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:29.447386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe34823-37c7-477f-9f46-248071234c0c None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:29.451281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 522/1059] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:42:29.454545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761158c-d4c6-4221-a084-ab515f049c52 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:29.455042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761158c-d4c6-4221-a084-ab515f049c52 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:29.455436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761158c-d4c6-4221-a084-ab515f049c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:29.472337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-5761158c-d4c6-4221-a084-ab515f049c52 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:29.473511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5761158c-d4c6-4221-a084-ab515f049c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:29.482004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b79a94-6841-4f11-93c4-cfc4f0b95e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:29.482668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b79a94-6841-4f11-93c4-cfc4f0b95e95 None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:29.482936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b79a94-6841-4f11-93c4-cfc4f0b95e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:29.483221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b79a94-6841-4f11-93c4-cfc4f0b95e95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:29.484007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b79a94-6841-4f11-93c4-cfc4f0b95e95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:29.485384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b79a94-6841-4f11-93c4-cfc4f0b95e95 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:29.507848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 538/1060] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 4 05:42:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:29.557522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 523/1061] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:29.581010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b79a94-6841-4f11-93c4-cfc4f0b95e95 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:29.687859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b79a94-6841-4f11-93c4-cfc4f0b95e95 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:29.724471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145af422-ca17-4fcb-b3d8-2ae9fa58fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:29.725297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145af422-ca17-4fcb-b3d8-2ae9fa58fc73 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:29.725576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145af422-ca17-4fcb-b3d8-2ae9fa58fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:29.725844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145af422-ca17-4fcb-b3d8-2ae9fa58fc73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:29.736115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145af422-ca17-4fcb-b3d8-2ae9fa58fc73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-145af422-ca17-4fcb-b3d8-2ae9fa58fc73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:29.744410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:29.745384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:29.746273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 539/1062] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 4 05:42:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:29.753762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761158c-d4c6-4221-a084-ab515f049c52 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:29.851407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:29.865416 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761158c-d4c6-4221-a084-ab515f049c52 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:29.900444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd48694-52fd-4ff8-8634-4003fb5088e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:29.901116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd48694-52fd-4ff8-8634-4003fb5088e5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:29.901399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd48694-52fd-4ff8-8634-4003fb5088e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:29.901674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd48694-52fd-4ff8-8634-4003fb5088e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:29.902430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd48694-52fd-4ff8-8634-4003fb5088e5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:29.902736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd48694-52fd-4ff8-8634-4003fb5088e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:29.904149 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd48694-52fd-4ff8-8634-4003fb5088e5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:29.912458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 524/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 4 05:42:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:29.918426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145af422-ca17-4fcb-b3d8-2ae9fa58fc73 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:29.938859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:30.024241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145af422-ca17-4fcb-b3d8-2ae9fa58fc73 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:30.058966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98ed50-cf51-4630-a81c-856e8ffcd077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:30.059803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98ed50-cf51-4630-a81c-856e8ffcd077 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.060081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98ed50-cf51-4630-a81c-856e8ffcd077 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.060354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98ed50-cf51-4630-a81c-856e8ffcd077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.071705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98ed50-cf51-4630-a81c-856e8ffcd077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:30.073175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98ed50-cf51-4630-a81c-856e8ffcd077 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-6c98ed50-cf51-4630-a81c-856e8ffcd077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:30.092192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:30.093111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:30.094022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:30.094022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:30.094022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:30.094022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:30.095423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 540/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:30.099704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd48694-52fd-4ff8-8634-4003fb5088e5 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:30.181994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efb4cf-f52e-48e9-9c4d-5717d98e1297 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.182182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efb4cf-f52e-48e9-9c4d-5717d98e1297 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.182425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efb4cf-f52e-48e9-9c4d-5717d98e1297 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.184085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 541/1065] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:42:30.188942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94fcc1-d391-4af5-9b92-ac7d43bcd9c8 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.189231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94fcc1-d391-4af5-9b92-ac7d43bcd9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.189492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94fcc1-d391-4af5-9b92-ac7d43bcd9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.207961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-eb94fcc1-d391-4af5-9b92-ac7d43bcd9c8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:30.208649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd48694-52fd-4ff8-8634-4003fb5088e5 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:30.209144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-eb94fcc1-d391-4af5-9b92-ac7d43bcd9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:30.244454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598dc90d-00a6-4c2e-ad1a-06fcbc89d4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:30.245138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dc90d-00a6-4c2e-ad1a-06fcbc89d4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.245421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dc90d-00a6-4c2e-ad1a-06fcbc89d4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.245691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dc90d-00a6-4c2e-ad1a-06fcbc89d4c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.246417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598dc90d-00a6-4c2e-ad1a-06fcbc89d4c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:30.246724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598dc90d-00a6-4c2e-ad1a-06fcbc89d4c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:30.248133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598dc90d-00a6-4c2e-ad1a-06fcbc89d4c3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:30.260139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 525/1066] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:30.265628 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598dc90d-00a6-4c2e-ad1a-06fcbc89d4c3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:30.279108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:30.285855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:30.290818 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 542/1067] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:30.313417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94fcc1-d391-4af5-9b92-ac7d43bcd9c8 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:30.376822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598dc90d-00a6-4c2e-ad1a-06fcbc89d4c3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:30.411124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94fcc1-d391-4af5-9b92-ac7d43bcd9c8 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:30.412550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a46b2-73f8-4d9e-bd52-8e8b3668139d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:30.413628 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a46b2-73f8-4d9e-bd52-8e8b3668139d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.413906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a46b2-73f8-4d9e-bd52-8e8b3668139d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.414178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a46b2-73f8-4d9e-bd52-8e8b3668139d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.421944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a46b2-73f8-4d9e-bd52-8e8b3668139d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:30.423567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a46b2-73f8-4d9e-bd52-8e8b3668139d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-cf4a46b2-73f8-4d9e-bd52-8e8b3668139d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:42:30.438942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:30.439905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:30.440857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 526/1068] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 4 05:42:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:30.446280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a46b2-73f8-4d9e-bd52-8e8b3668139d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:30.449881 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698ebf0-5980-4ef4-952c-43c5995e5b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:30.450671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698ebf0-5980-4ef4-952c-43c5995e5b7b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.450945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698ebf0-5980-4ef4-952c-43c5995e5b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.451214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698ebf0-5980-4ef4-952c-43c5995e5b7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.462153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698ebf0-5980-4ef4-952c-43c5995e5b7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-1698ebf0-5980-4ef4-952c-43c5995e5b7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:30.469888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:30.470958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:30.471957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 543/1069] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:30.477255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698ebf0-5980-4ef4-952c-43c5995e5b7b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:30.523814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:30.544316 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:30.556127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a46b2-73f8-4d9e-bd52-8e8b3668139d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:30.587834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698ebf0-5980-4ef4-952c-43c5995e5b7b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:30.590877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ceabe-2bbd-4ed7-bd14-9849a9310425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:30.591595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ceabe-2bbd-4ed7-bd14-9849a9310425 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.591873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ceabe-2bbd-4ed7-bd14-9849a9310425 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.592141 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ceabe-2bbd-4ed7-bd14-9849a9310425 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.592980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ceabe-2bbd-4ed7-bd14-9849a9310425 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:30.593287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ceabe-2bbd-4ed7-bd14-9849a9310425 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:30.594728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ceabe-2bbd-4ed7-bd14-9849a9310425 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:30.611883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 527/1070] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 4 05:42:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:30.617397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ceabe-2bbd-4ed7-bd14-9849a9310425 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:30.628782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8176e44-ecfb-4850-9d48-ef4ccc05beb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:30.629867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8176e44-ecfb-4850-9d48-ef4ccc05beb4 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.630122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8176e44-ecfb-4850-9d48-ef4ccc05beb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.630745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8176e44-ecfb-4850-9d48-ef4ccc05beb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.631570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8176e44-ecfb-4850-9d48-ef4ccc05beb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:30.631853 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8176e44-ecfb-4850-9d48-ef4ccc05beb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:30.633292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8176e44-ecfb-4850-9d48-ef4ccc05beb4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:30.643790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 544/1071] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:42:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:30.649923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8176e44-ecfb-4850-9d48-ef4ccc05beb4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:30.738119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ceabe-2bbd-4ed7-bd14-9849a9310425 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:30.758302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8176e44-ecfb-4850-9d48-ef4ccc05beb4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:30.765165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:30.775202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0091d4a-5eac-4817-8f2c-fb97ade64b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:30.776150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0091d4a-5eac-4817-8f2c-fb97ade64b5d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.776426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0091d4a-5eac-4817-8f2c-fb97ade64b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.776797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0091d4a-5eac-4817-8f2c-fb97ade64b5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.782332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:30.789866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0091d4a-5eac-4817-8f2c-fb97ade64b5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:30.791367 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0091d4a-5eac-4817-8f2c-fb97ade64b5d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:30.794884 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484e456-49c6-4f0a-b14a-964f89902419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:30.795791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484e456-49c6-4f0a-b14a-964f89902419 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.796053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484e456-49c6-4f0a-b14a-964f89902419 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.796322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484e456-49c6-4f0a-b14a-964f89902419 None admin] PATH_INFO: `/v3/users/4eb9fc26b3bd4c64bd4453df1c8e8438` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.797534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484e456-49c6-4f0a-b14a-964f89902419 None admin] RBAC: Authorizing `identity:get_user(user_id=4eb9fc26b3bd4c64bd4453df1c8e8438)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:30.798936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484e456-49c6-4f0a-b14a-964f89902419 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:30.805868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 545/1072] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:30 2022] GET /identity/v3/users/4eb9fc26b3bd4c64bd4453df1c8e8438 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:30.807108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484e456-49c6-4f0a-b14a-964f89902419 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-b0091d4a-5eac-4817-8f2c-fb97ade64b5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:30.812587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:30.813486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:30.814432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:30.814432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:30.814432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:30.814432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:30.814432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 528/1073] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:30.820539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0091d4a-5eac-4817-8f2c-fb97ade64b5d None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:30.925665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484e456-49c6-4f0a-b14a-964f89902419 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:30.929412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0091d4a-5eac-4817-8f2c-fb97ade64b5d None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:30.964653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ed86c-03e6-447f-a06e-34821347d6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:30.965523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ed86c-03e6-447f-a06e-34821347d6ef None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.965797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ed86c-03e6-447f-a06e-34821347d6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.966064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ed86c-03e6-447f-a06e-34821347d6ef None admin] PATH_INFO: `/v3/projects/a48d4d741809450a82825e526404656c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.967306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ed86c-03e6-447f-a06e-34821347d6ef None admin] RBAC: Authorizing `identity:get_project(project_id=a48d4d741809450a82825e526404656c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:30.967632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e17e4-a4c0-4347-aaa3-2a26b19d8a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:30.968308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e17e4-a4c0-4347-aaa3-2a26b19d8a86 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:30.968580 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e17e4-a4c0-4347-aaa3-2a26b19d8a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:30.968750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ed86c-03e6-447f-a06e-34821347d6ef None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:30.968857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e17e4-a4c0-4347-aaa3-2a26b19d8a86 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:30.969572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e17e4-a4c0-4347-aaa3-2a26b19d8a86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:30.969876 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e17e4-a4c0-4347-aaa3-2a26b19d8a86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:30.970622 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 546/1074] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:42:30 2022] GET /identity/v3/projects/a48d4d741809450a82825e526404656c => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:30.971635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e17e4-a4c0-4347-aaa3-2a26b19d8a86 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:30.976639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ed86c-03e6-447f-a06e-34821347d6ef None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:30.989049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 529/1075] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:30.990068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e17e4-a4c0-4347-aaa3-2a26b19d8a86 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:31.034453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:31.042112 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:31.089944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ed86c-03e6-447f-a06e-34821347d6ef None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:31.111864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e17e4-a4c0-4347-aaa3-2a26b19d8a86 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:31.138070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477fbdb-3542-4bf4-99a0-931da98c5ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:31.139117 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477fbdb-3542-4bf4-99a0-931da98c5ac8 None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:31.139350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477fbdb-3542-4bf4-99a0-931da98c5ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:31.139629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477fbdb-3542-4bf4-99a0-931da98c5ac8 None admin] PATH_INFO: `/v3/projects/a48d4d741809450a82825e526404656c/users/4eb9fc26b3bd4c64bd4453df1c8e8438/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:31.141698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477fbdb-3542-4bf4-99a0-931da98c5ac8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a48d4d741809450a82825e526404656c, user_id=4eb9fc26b3bd4c64bd4453df1c8e8438, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:31.143398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477fbdb-3542-4bf4-99a0-931da98c5ac8 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:31.149376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82914ce5-c712-4f1b-ae82-8637ef00488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:31.150457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82914ce5-c712-4f1b-ae82-8637ef00488c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:31.150866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82914ce5-c712-4f1b-ae82-8637ef00488c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:31.151309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82914ce5-c712-4f1b-ae82-8637ef00488c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:31.160018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82914ce5-c712-4f1b-ae82-8637ef00488c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:31.160957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 547/1076] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:42:30 2022] PUT /identity/v3/projects/a48d4d741809450a82825e526404656c/users/4eb9fc26b3bd4c64bd4453df1c8e8438/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:42:31.161492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82914ce5-c712-4f1b-ae82-8637ef00488c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-82914ce5-c712-4f1b-ae82-8637ef00488c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:42:31.178774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:31.180220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:31.181445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:42:31.181445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:42:31.181445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:42:31.181445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:42:31.181445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:42:31.181445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:42:31.181445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:31.181445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:31.185470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 530/1077] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:31.187033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477fbdb-3542-4bf4-99a0-931da98c5ac8 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:31.310676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477fbdb-3542-4bf4-99a0-931da98c5ac8 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:31.337516 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:31.353900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059ef65-bb21-43a9-aa4e-1e191248f2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:31.354849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059ef65-bb21-43a9-aa4e-1e191248f2ad None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:31.355127 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059ef65-bb21-43a9-aa4e-1e191248f2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:31.355431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059ef65-bb21-43a9-aa4e-1e191248f2ad None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:31.356234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059ef65-bb21-43a9-aa4e-1e191248f2ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:31.356540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059ef65-bb21-43a9-aa4e-1e191248f2ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:31.358014 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059ef65-bb21-43a9-aa4e-1e191248f2ad None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:31.375319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 548/1078] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:31.379058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82914ce5-c712-4f1b-ae82-8637ef00488c None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:31.500776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:31.505117 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82914ce5-c712-4f1b-ae82-8637ef00488c None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:31.553660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6466b11-c3f9-4c7f-9966-35dffbabb4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:31.554574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6466b11-c3f9-4c7f-9966-35dffbabb4e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:31.554857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6466b11-c3f9-4c7f-9966-35dffbabb4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:31.555130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6466b11-c3f9-4c7f-9966-35dffbabb4e0 None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/2cb245e4300e4d5fbdd425e936f4ac77/roles/85e17d8dc6cb480abe4e2bb231995a79` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:31.557514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6466b11-c3f9-4c7f-9966-35dffbabb4e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=2cb245e4300e4d5fbdd425e936f4ac77, role_id=85e17d8dc6cb480abe4e2bb231995a79)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:31.558967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6466b11-c3f9-4c7f-9966-35dffbabb4e0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:31.574670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 531/1079] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:42:31 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/2cb245e4300e4d5fbdd425e936f4ac77/roles/85e17d8dc6cb480abe4e2bb231995a79 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:42:31.606349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356cf7c-3829-41c7-bdaf-ab70ddb2eacf None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:31.606859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356cf7c-3829-41c7-bdaf-ab70ddb2eacf None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:31.607274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356cf7c-3829-41c7-bdaf-ab70ddb2eacf None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:31.608787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 549/1080] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:31.614850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772430db-1de3-4905-bb86-9fe82b580b57 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:31.615195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772430db-1de3-4905-bb86-9fe82b580b57 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:31.615461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772430db-1de3-4905-bb86-9fe82b580b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:31.634412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-772430db-1de3-4905-bb86-9fe82b580b57 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:31.635759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-772430db-1de3-4905-bb86-9fe82b580b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:31.727682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 532/1081] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:31.750300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356cf7c-3829-41c7-bdaf-ab70ddb2eacf None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:31.862765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356cf7c-3829-41c7-bdaf-ab70ddb2eacf None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:31.899730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6111b-6ce8-486f-9333-45e9ec9ef489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:31.900395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6111b-6ce8-486f-9333-45e9ec9ef489 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:31.900679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6111b-6ce8-486f-9333-45e9ec9ef489 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:31.900954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6111b-6ce8-486f-9333-45e9ec9ef489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:31.948387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:31.971813 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6111b-6ce8-486f-9333-45e9ec9ef489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:31.973351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6111b-6ce8-486f-9333-45e9ec9ef489 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:32.079282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 550/1082] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:42:32.081276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772430db-1de3-4905-bb86-9fe82b580b57 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:32.176459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:32.188442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772430db-1de3-4905-bb86-9fe82b580b57 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:32.223958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb50af-e4a5-4869-921a-ee724d41a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:32.224772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb50af-e4a5-4869-921a-ee724d41a743 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:32.225048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb50af-e4a5-4869-921a-ee724d41a743 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:32.225316 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb50af-e4a5-4869-921a-ee724d41a743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:32.235649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb50af-e4a5-4869-921a-ee724d41a743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:32.237098 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb50af-e4a5-4869-921a-ee724d41a743 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-6ddb50af-e4a5-4869-921a-ee724d41a743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:32.254640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:32.255618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:32.256521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:32.256521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:32.256521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:32.256521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:32.259393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 533/1083] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:32.262007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6111b-6ce8-486f-9333-45e9ec9ef489 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:32.345008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:32.370121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6111b-6ce8-486f-9333-45e9ec9ef489 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:32.405484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cde641-dccd-438d-b51e-69bd84e7c9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:32.406408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cde641-dccd-438d-b51e-69bd84e7c9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:32.406835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cde641-dccd-438d-b51e-69bd84e7c9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:32.407281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cde641-dccd-438d-b51e-69bd84e7c9d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:32.408167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cde641-dccd-438d-b51e-69bd84e7c9d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:32.408618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cde641-dccd-438d-b51e-69bd84e7c9d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:32.410137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cde641-dccd-438d-b51e-69bd84e7c9d6 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:32.422306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 551/1084] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:33.217012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2f8d-59e7-4797-8692-f3053028875b None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:33.217585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2f8d-59e7-4797-8692-f3053028875b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:33.217585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2f8d-59e7-4797-8692-f3053028875b None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:33.218964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 534/1085] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:33.224411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163814da-17ee-494c-9444-6577897e3952 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:33.224790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163814da-17ee-494c-9444-6577897e3952 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:33.224965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163814da-17ee-494c-9444-6577897e3952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:33.242728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-163814da-17ee-494c-9444-6577897e3952 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:33.244091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-163814da-17ee-494c-9444-6577897e3952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:33.313349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 552/1086] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:33.337370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df2f8d-59e7-4797-8692-f3053028875b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:33.445339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df2f8d-59e7-4797-8692-f3053028875b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:33.479886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8279c-cb0a-4ced-97db-ef40f1dd017d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:33.480692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8279c-cb0a-4ced-97db-ef40f1dd017d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:33.480975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8279c-cb0a-4ced-97db-ef40f1dd017d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:33.481246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8279c-cb0a-4ced-97db-ef40f1dd017d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:33.491634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8279c-cb0a-4ced-97db-ef40f1dd017d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-e1e8279c-cb0a-4ced-97db-ef40f1dd017d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:33.499376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:33.500326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:33.501173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:33.503767 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 535/1087] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:42:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:33.506790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163814da-17ee-494c-9444-6577897e3952 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:33.575115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:33.610598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163814da-17ee-494c-9444-6577897e3952 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:33.618260 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dd81e-60ca-4323-b016-db173ce8d34f None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:33.618517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dd81e-60ca-4323-b016-db173ce8d34f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:33.618819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dd81e-60ca-4323-b016-db173ce8d34f None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:33.620232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 536/1088] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:33.625080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46777b-ece4-487a-bc14-d430739baa09 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:33.625474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46777b-ece4-487a-bc14-d430739baa09 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:33.625756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46777b-ece4-487a-bc14-d430739baa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:33.631729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:33.644749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-7a46777b-ece4-487a-bc14-d430739baa09 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:33.645609 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83373b-1e3e-4afa-b852-e4e6c5e54c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:33.645937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7a46777b-ece4-487a-bc14-d430739baa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:33.646273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83373b-1e3e-4afa-b852-e4e6c5e54c7a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:33.646549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83373b-1e3e-4afa-b852-e4e6c5e54c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:33.646914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83373b-1e3e-4afa-b852-e4e6c5e54c7a None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:33.647690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83373b-1e3e-4afa-b852-e4e6c5e54c7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:33.647990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83373b-1e3e-4afa-b852-e4e6c5e54c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:33.649388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83373b-1e3e-4afa-b852-e4e6c5e54c7a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:33.657696 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 553/1089] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:42:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:33.663440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83373b-1e3e-4afa-b852-e4e6c5e54c7a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:33.710908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 537/1090] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:33.734424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a46777b-ece4-487a-bc14-d430739baa09 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:33.765686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83373b-1e3e-4afa-b852-e4e6c5e54c7a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:33.796944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:33.804440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe50d79-3206-402a-84cd-d19d4f9b9231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:33.805332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe50d79-3206-402a-84cd-d19d4f9b9231 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:33.805617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe50d79-3206-402a-84cd-d19d4f9b9231 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:33.805895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe50d79-3206-402a-84cd-d19d4f9b9231 None admin] PATH_INFO: `/v3/users/4eb9fc26b3bd4c64bd4453df1c8e8438` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:33.807136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe50d79-3206-402a-84cd-d19d4f9b9231 None admin] RBAC: Authorizing `identity:get_user(user_id=4eb9fc26b3bd4c64bd4453df1c8e8438)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:33.808623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe50d79-3206-402a-84cd-d19d4f9b9231 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:33.810280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 554/1091] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:33 2022] GET /identity/v3/users/4eb9fc26b3bd4c64bd4453df1c8e8438 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:33.816063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe50d79-3206-402a-84cd-d19d4f9b9231 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:33.838963 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a46777b-ece4-487a-bc14-d430739baa09 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:33.867546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:33.874421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b696d-d342-4e99-9801-f6505c448e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:33.875266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b696d-d342-4e99-9801-f6505c448e63 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:33.875534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b696d-d342-4e99-9801-f6505c448e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:33.875805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b696d-d342-4e99-9801-f6505c448e63 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:33.886046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b696d-d342-4e99-9801-f6505c448e63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-8d4b696d-d342-4e99-9801-f6505c448e63 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:33.894353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:33.895511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:33.896568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 538/1092] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 4 05:42:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:33.901730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b696d-d342-4e99-9801-f6505c448e63 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:33.918139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe50d79-3206-402a-84cd-d19d4f9b9231 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:33.951940 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e80ca0-e1ab-4013-bc37-018d0288bf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:33.952587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e80ca0-e1ab-4013-bc37-018d0288bf26 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:33.952867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e80ca0-e1ab-4013-bc37-018d0288bf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:33.953128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e80ca0-e1ab-4013-bc37-018d0288bf26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:33.998232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b696d-d342-4e99-9801-f6505c448e63 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:34.017125 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:34.020598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e80ca0-e1ab-4013-bc37-018d0288bf26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:34.022203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e80ca0-e1ab-4013-bc37-018d0288bf26 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:34.032665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ed30e-b758-482e-b0d0-099f9c5d2cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:34.033333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ed30e-b758-482e-b0d0-099f9c5d2cea None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:34.033682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ed30e-b758-482e-b0d0-099f9c5d2cea None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:34.033973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ed30e-b758-482e-b0d0-099f9c5d2cea None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:34.034718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ed30e-b758-482e-b0d0-099f9c5d2cea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:34.035001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ed30e-b758-482e-b0d0-099f9c5d2cea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:34.036395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ed30e-b758-482e-b0d0-099f9c5d2cea None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:34.044496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 539/1093] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 4 05:42:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:34.050231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ed30e-b758-482e-b0d0-099f9c5d2cea None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:34.105170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:34.124123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 555/1094] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 05:42:34.130112 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e80ca0-e1ab-4013-bc37-018d0288bf26 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:34.149424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ed30e-b758-482e-b0d0-099f9c5d2cea None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:34.183546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833f4e9-de65-4f85-847d-d942be2b68fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:34.184339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833f4e9-de65-4f85-847d-d942be2b68fc None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:34.184635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833f4e9-de65-4f85-847d-d942be2b68fc None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:34.184897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833f4e9-de65-4f85-847d-d942be2b68fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:34.195740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833f4e9-de65-4f85-847d-d942be2b68fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:34.197239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833f4e9-de65-4f85-847d-d942be2b68fc None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-a833f4e9-de65-4f85-847d-d942be2b68fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:34.214923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:34.216005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:34.216905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:34.216905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:34.216905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:34.216905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:34.216905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 540/1095] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:34.221971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833f4e9-de65-4f85-847d-d942be2b68fc None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:34.224259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:34.228629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e80ca0-e1ab-4013-bc37-018d0288bf26 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:34.265439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc499b-4bf9-4ecc-83dd-6ee34aa89a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:34.266232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc499b-4bf9-4ecc-83dd-6ee34aa89a0d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:34.266512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc499b-4bf9-4ecc-83dd-6ee34aa89a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:34.266778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc499b-4bf9-4ecc-83dd-6ee34aa89a0d None admin] PATH_INFO: `/v3/projects/a48d4d741809450a82825e526404656c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:34.267969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc499b-4bf9-4ecc-83dd-6ee34aa89a0d None admin] RBAC: Authorizing `identity:get_project(project_id=a48d4d741809450a82825e526404656c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:34.269423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc499b-4bf9-4ecc-83dd-6ee34aa89a0d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:34.271119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 556/1096] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:42:34 2022] GET /identity/v3/projects/a48d4d741809450a82825e526404656c => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:34.277224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc499b-4bf9-4ecc-83dd-6ee34aa89a0d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:34.322945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833f4e9-de65-4f85-847d-d942be2b68fc None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:34.329939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:34.358303 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa963f8-a5e2-427c-b39f-a916d639c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:34.358970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa963f8-a5e2-427c-b39f-a916d639c248 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:34.359257 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa963f8-a5e2-427c-b39f-a916d639c248 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:34.359562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa963f8-a5e2-427c-b39f-a916d639c248 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:34.360286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa963f8-a5e2-427c-b39f-a916d639c248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:34.360592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa963f8-a5e2-427c-b39f-a916d639c248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:34.361967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa963f8-a5e2-427c-b39f-a916d639c248 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:34.373240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 541/1097] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:34.374851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc499b-4bf9-4ecc-83dd-6ee34aa89a0d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:34.378885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa963f8-a5e2-427c-b39f-a916d639c248 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:34.409719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920f62f-f427-4903-ab67-f19995eee4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:34.410387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920f62f-f427-4903-ab67-f19995eee4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:34.410664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920f62f-f427-4903-ab67-f19995eee4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:34.410937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920f62f-f427-4903-ab67-f19995eee4f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:34.411819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920f62f-f427-4903-ab67-f19995eee4f8 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=a48d4d741809450a82825e526404656c, user.id=4eb9fc26b3bd4c64bd4453df1c8e8438) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:34.412130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920f62f-f427-4903-ab67-f19995eee4f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:34.413588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920f62f-f427-4903-ab67-f19995eee4f8 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:34.419402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 557/1098] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:34 2022] GET /identity/v3/role_assignments?user.id=4eb9fc26b3bd4c64bd4453df1c8e8438&scope.project.id=a48d4d741809450a82825e526404656c&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:34.480746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa963f8-a5e2-427c-b39f-a916d639c248 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:34.521931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53ff1d-2df1-4ab3-84b2-566ed3ccfc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:34.522753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53ff1d-2df1-4ab3-84b2-566ed3ccfc88 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:34.523036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53ff1d-2df1-4ab3-84b2-566ed3ccfc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:34.523341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53ff1d-2df1-4ab3-84b2-566ed3ccfc88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:34.532674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53ff1d-2df1-4ab3-84b2-566ed3ccfc88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:34.534155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53ff1d-2df1-4ab3-84b2-566ed3ccfc88 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-0b53ff1d-2df1-4ab3-84b2-566ed3ccfc88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:42:34.550139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:34.551282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:34.552267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 542/1099] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 4 05:42:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:42:34.557822 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920f62f-f427-4903-ab67-f19995eee4f8 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:34.593017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:34.664395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920f62f-f427-4903-ab67-f19995eee4f8 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:34.702247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518b563-d8e0-43c5-a7f1-ac4ee1b3337d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:34.702923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518b563-d8e0-43c5-a7f1-ac4ee1b3337d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:34.703203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518b563-d8e0-43c5-a7f1-ac4ee1b3337d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:34.703503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518b563-d8e0-43c5-a7f1-ac4ee1b3337d None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:34.704347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518b563-d8e0-43c5-a7f1-ac4ee1b3337d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:34.704657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518b563-d8e0-43c5-a7f1-ac4ee1b3337d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:34.706144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518b563-d8e0-43c5-a7f1-ac4ee1b3337d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:34.724383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 558/1100] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 4 05:42:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:34.731183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53ff1d-2df1-4ab3-84b2-566ed3ccfc88 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:34.760080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:34.837669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53ff1d-2df1-4ab3-84b2-566ed3ccfc88 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:34.873113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c76b94-a715-4286-8d08-558da80ec8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:34.874038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c76b94-a715-4286-8d08-558da80ec8bb None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:34.874328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c76b94-a715-4286-8d08-558da80ec8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:34.874604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c76b94-a715-4286-8d08-558da80ec8bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:34.944326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c76b94-a715-4286-8d08-558da80ec8bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:34.945794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c76b94-a715-4286-8d08-558da80ec8bb None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:34.971480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:35.045731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 543/1101] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:42:35.052448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518b563-d8e0-43c5-a7f1-ac4ee1b3337d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:35.143364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:35.157564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518b563-d8e0-43c5-a7f1-ac4ee1b3337d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:35.194095 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81e54-3600-4736-9ef6-b1bbbd984246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:35.194903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81e54-3600-4736-9ef6-b1bbbd984246 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:35.195180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81e54-3600-4736-9ef6-b1bbbd984246 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:35.195509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81e54-3600-4736-9ef6-b1bbbd984246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:35.206239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa81e54-3600-4736-9ef6-b1bbbd984246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:35.207747 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa81e54-3600-4736-9ef6-b1bbbd984246 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-6aa81e54-3600-4736-9ef6-b1bbbd984246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:42:35.228272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:35.230620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:35.232456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:42:35.232456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:42:35.232456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:42:35.232456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:35.232456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 559/1102] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:35.236726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c76b94-a715-4286-8d08-558da80ec8bb None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:35.336834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c76b94-a715-4286-8d08-558da80ec8bb None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:35.362616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:35.372461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897219a8-70e5-43a8-a273-23600bd33a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:35.373135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897219a8-70e5-43a8-a273-23600bd33a82 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:35.373419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897219a8-70e5-43a8-a273-23600bd33a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:35.373700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897219a8-70e5-43a8-a273-23600bd33a82 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:35.374431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897219a8-70e5-43a8-a273-23600bd33a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:35.374737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897219a8-70e5-43a8-a273-23600bd33a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:35.376196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897219a8-70e5-43a8-a273-23600bd33a82 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:35.387690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 544/1103] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:35.393480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81e54-3600-4736-9ef6-b1bbbd984246 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:35.494465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81e54-3600-4736-9ef6-b1bbbd984246 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:35.533901 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e937d7-88be-46cf-831b-a9b625acad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:35.534736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e937d7-88be-46cf-831b-a9b625acad14 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:35.535017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e937d7-88be-46cf-831b-a9b625acad14 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:35.535308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e937d7-88be-46cf-831b-a9b625acad14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:35.543276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e937d7-88be-46cf-831b-a9b625acad14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:35.544789 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e937d7-88be-46cf-831b-a9b625acad14 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:35.547826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-d5e937d7-88be-46cf-831b-a9b625acad14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:42:35.561198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:35.562294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:35.563206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:42:35.563206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:42:35.563206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:42:35.563206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:42:35.563206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:42:35.563206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:42:35.563206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:42:35.563206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:35.563206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 560/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:35.568373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897219a8-70e5-43a8-a273-23600bd33a82 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:35.674040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897219a8-70e5-43a8-a273-23600bd33a82 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:35.708385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f5778-5b88-4e64-ae0e-879955aceff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:35.709049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f5778-5b88-4e64-ae0e-879955aceff8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:35.709327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f5778-5b88-4e64-ae0e-879955aceff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:35.709603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f5778-5b88-4e64-ae0e-879955aceff8 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:35.710375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f5778-5b88-4e64-ae0e-879955aceff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:35.710684 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f5778-5b88-4e64-ae0e-879955aceff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:35.712348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f5778-5b88-4e64-ae0e-879955aceff8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:35.724259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 545/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:35.730501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e937d7-88be-46cf-831b-a9b625acad14 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:35.835491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e937d7-88be-46cf-831b-a9b625acad14 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:35.870707 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ac2b1-9bc5-4b1e-9c27-2a18021f0e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:35.871383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac2b1-9bc5-4b1e-9c27-2a18021f0e9e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:35.871662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac2b1-9bc5-4b1e-9c27-2a18021f0e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:35.871936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac2b1-9bc5-4b1e-9c27-2a18021f0e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:35.872762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac2b1-9bc5-4b1e-9c27-2a18021f0e9e None admin] RBAC: Adding query filter params (role.id=85e17d8dc6cb480abe4e2bb231995a79, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=2cb245e4300e4d5fbdd425e936f4ac77) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:35.873070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac2b1-9bc5-4b1e-9c27-2a18021f0e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:35.874523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac2b1-9bc5-4b1e-9c27-2a18021f0e9e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:35.880679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 561/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:35 2022] GET /identity/v3/role_assignments?user.id=2cb245e4300e4d5fbdd425e936f4ac77&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=85e17d8dc6cb480abe4e2bb231995a79 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:36.461035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d205a-9dc5-4fb0-9cdd-394852bdb3d9 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:36.461558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d205a-9dc5-4fb0-9cdd-394852bdb3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:36.461558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d205a-9dc5-4fb0-9cdd-394852bdb3d9 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:36.463133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 546/1107] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:36.468734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fc8f0-43f8-470b-bbb7-47cedd17d032 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:36.469201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fc8f0-43f8-470b-bbb7-47cedd17d032 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:36.469302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fc8f0-43f8-470b-bbb7-47cedd17d032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:36.488173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-9b0fc8f0-43f8-470b-bbb7-47cedd17d032 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:36.489365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9b0fc8f0-43f8-470b-bbb7-47cedd17d032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:36.552867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:36.562181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 562/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:36.587838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d205a-9dc5-4fb0-9cdd-394852bdb3d9 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:36.639780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:36.694460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d205a-9dc5-4fb0-9cdd-394852bdb3d9 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:36.734838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b61667-76c1-4092-a596-4465176c0b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:36.734838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b61667-76c1-4092-a596-4465176c0b6d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:36.735176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b61667-76c1-4092-a596-4465176c0b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:36.735294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b61667-76c1-4092-a596-4465176c0b6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:36.736457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b61667-76c1-4092-a596-4465176c0b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:36.737891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b61667-76c1-4092-a596-4465176c0b6d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:36.739563 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 547/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:36.745513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fc8f0-43f8-470b-bbb7-47cedd17d032 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:36.848188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fc8f0-43f8-470b-bbb7-47cedd17d032 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:36.886993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a24c0-302a-4fa7-8435-f1211d400c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:36.887705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a24c0-302a-4fa7-8435-f1211d400c5a None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:36.887985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a24c0-302a-4fa7-8435-f1211d400c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:36.888257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a24c0-302a-4fa7-8435-f1211d400c5a None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:36.888979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a24c0-302a-4fa7-8435-f1211d400c5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:36.890433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a24c0-302a-4fa7-8435-f1211d400c5a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:36.914454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 563/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:42:36 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:38.956285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303ff48-9dbb-4c2e-8b9d-70f127269452 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:38.956819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303ff48-9dbb-4c2e-8b9d-70f127269452 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:38.956819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303ff48-9dbb-4c2e-8b9d-70f127269452 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:38.958123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 548/1111] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:38.964486 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0663a-1ce8-405d-8ce2-fb7935d493ca None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:38.965181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0663a-1ce8-405d-8ce2-fb7935d493ca None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:38.965368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0663a-1ce8-405d-8ce2-fb7935d493ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:38.988499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-69c0663a-1ce8-405d-8ce2-fb7935d493ca None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:38.989704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-69c0663a-1ce8-405d-8ce2-fb7935d493ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:39.041566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:39.074446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 564/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:39.098249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303ff48-9dbb-4c2e-8b9d-70f127269452 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:39.206080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303ff48-9dbb-4c2e-8b9d-70f127269452 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:39.220929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:39.246012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c45b1-4862-4629-873d-a9314c267ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:39.246872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c45b1-4862-4629-873d-a9314c267ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:39.247152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c45b1-4862-4629-873d-a9314c267ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:39.247459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c45b1-4862-4629-873d-a9314c267ca9 None admin] PATH_INFO: `/v3/roles/d2054df5a6d24f5d90793b9b81cd4b4b` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:39.249171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c45b1-4862-4629-873d-a9314c267ca9 None admin] RBAC: Authorizing `identity:get_role(role_id=d2054df5a6d24f5d90793b9b81cd4b4b)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:39.250788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c45b1-4862-4629-873d-a9314c267ca9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:39.252321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 549/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:39 2022] GET /identity/v3/roles/d2054df5a6d24f5d90793b9b81cd4b4b => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:39.261084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0663a-1ce8-405d-8ce2-fb7935d493ca None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:39.366842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0663a-1ce8-405d-8ce2-fb7935d493ca None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:39.411032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764aa174-7eba-48d9-a2ff-902d9925567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:39.411953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764aa174-7eba-48d9-a2ff-902d9925567b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:39.412236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764aa174-7eba-48d9-a2ff-902d9925567b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:39.412506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764aa174-7eba-48d9-a2ff-902d9925567b None admin] PATH_INFO: `/v3/users/9f811f21c78343e3b16a4a859796bf0e` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:39.427644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764aa174-7eba-48d9-a2ff-902d9925567b None admin] RBAC: Authorizing `identity:get_user(user_id=9f811f21c78343e3b16a4a859796bf0e)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:39.429115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764aa174-7eba-48d9-a2ff-902d9925567b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:39.430858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 565/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:39 2022] GET /identity/v3/users/9f811f21c78343e3b16a4a859796bf0e => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:39.440522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c45b1-4862-4629-873d-a9314c267ca9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:39.558415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c45b1-4862-4629-873d-a9314c267ca9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:39.599829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ea515-a537-42e1-b3d9-f930220ad928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:39.600480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ea515-a537-42e1-b3d9-f930220ad928 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:39.600764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ea515-a537-42e1-b3d9-f930220ad928 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:39.601041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ea515-a537-42e1-b3d9-f930220ad928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:39.652129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:39.671928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ea515-a537-42e1-b3d9-f930220ad928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:39.673374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ea515-a537-42e1-b3d9-f930220ad928 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:39.773321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 550/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:42:39.780086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764aa174-7eba-48d9-a2ff-902d9925567b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:39.826443 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:39.888597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764aa174-7eba-48d9-a2ff-902d9925567b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:39.926113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc521726-9e76-4977-a240-50856fe2fa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:39.926963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc521726-9e76-4977-a240-50856fe2fa5d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:39.927236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc521726-9e76-4977-a240-50856fe2fa5d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:39.927542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc521726-9e76-4977-a240-50856fe2fa5d None admin] PATH_INFO: `/v3/projects/a48d4d741809450a82825e526404656c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:39.928741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc521726-9e76-4977-a240-50856fe2fa5d None admin] RBAC: Authorizing `identity:get_project(project_id=a48d4d741809450a82825e526404656c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:39.930179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc521726-9e76-4977-a240-50856fe2fa5d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:39.932072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 566/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:42:39 2022] GET /identity/v3/projects/a48d4d741809450a82825e526404656c => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:39.941406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ea515-a537-42e1-b3d9-f930220ad928 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:40.030434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:40.051756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ea515-a537-42e1-b3d9-f930220ad928 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:40.088880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96812d32-89f8-465c-ae56-090a82ae48c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:40.089534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96812d32-89f8-465c-ae56-090a82ae48c2 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:40.089815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96812d32-89f8-465c-ae56-090a82ae48c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:40.090087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96812d32-89f8-465c-ae56-090a82ae48c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:40.090925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96812d32-89f8-465c-ae56-090a82ae48c2 None admin] RBAC: Adding query filter params (role.id=d2054df5a6d24f5d90793b9b81cd4b4b, scope.project.id=a48d4d741809450a82825e526404656c, user.id=9f811f21c78343e3b16a4a859796bf0e) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:40.091236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96812d32-89f8-465c-ae56-090a82ae48c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:40.092714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96812d32-89f8-465c-ae56-090a82ae48c2 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:40.098305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 551/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:39 2022] GET /identity/v3/role_assignments?user.id=9f811f21c78343e3b16a4a859796bf0e&scope.project.id=a48d4d741809450a82825e526404656c&role.id=d2054df5a6d24f5d90793b9b81cd4b4b => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:42.093414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f3828-bcec-4dfd-a85d-123af16a27c2 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:42.094271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f3828-bcec-4dfd-a85d-123af16a27c2 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:42.094271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f3828-bcec-4dfd-a85d-123af16a27c2 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:42.096586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 567/1118] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:42.103204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434a509-bb09-4709-a853-01d0497524ac None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:42.103661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434a509-bb09-4709-a853-01d0497524ac None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:42.103750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434a509-bb09-4709-a853-01d0497524ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:42.129525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-0434a509-bb09-4709-a853-01d0497524ac None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:42.131487 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0434a509-bb09-4709-a853-01d0497524ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:42.202657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 552/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:42.226188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f3828-bcec-4dfd-a85d-123af16a27c2 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:42.352065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f3828-bcec-4dfd-a85d-123af16a27c2 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:42.359507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:42.390927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7ff96-0b75-4bc0-9a3d-745ef8cddc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:42.391820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7ff96-0b75-4bc0-9a3d-745ef8cddc1a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:42.392105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7ff96-0b75-4bc0-9a3d-745ef8cddc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:42.392380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7ff96-0b75-4bc0-9a3d-745ef8cddc1a None admin] PATH_INFO: `/v3/roles/d2054df5a6d24f5d90793b9b81cd4b4b` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:42.393963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7ff96-0b75-4bc0-9a3d-745ef8cddc1a None admin] RBAC: Authorizing `identity:get_role(role_id=d2054df5a6d24f5d90793b9b81cd4b4b)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:42.395393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7ff96-0b75-4bc0-9a3d-745ef8cddc1a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:42.396780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 568/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:42 2022] GET /identity/v3/roles/d2054df5a6d24f5d90793b9b81cd4b4b => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:42.403074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434a509-bb09-4709-a853-01d0497524ac None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:42.488633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:42.514816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434a509-bb09-4709-a853-01d0497524ac None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:42.547967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314abae-ce3c-4ec5-8ffb-73103ed7cc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:42.548855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314abae-ce3c-4ec5-8ffb-73103ed7cc8e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:42.549136 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314abae-ce3c-4ec5-8ffb-73103ed7cc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:42.549413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314abae-ce3c-4ec5-8ffb-73103ed7cc8e None admin] PATH_INFO: `/v3/users/9f811f21c78343e3b16a4a859796bf0e` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:42.550709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314abae-ce3c-4ec5-8ffb-73103ed7cc8e None admin] RBAC: Authorizing `identity:get_user(user_id=9f811f21c78343e3b16a4a859796bf0e)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:42.552181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314abae-ce3c-4ec5-8ffb-73103ed7cc8e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:42.553812 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 553/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:42 2022] GET /identity/v3/users/9f811f21c78343e3b16a4a859796bf0e => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:42.563272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7ff96-0b75-4bc0-9a3d-745ef8cddc1a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:42.667588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7ff96-0b75-4bc0-9a3d-745ef8cddc1a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:42.704544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b088c-f829-43a8-ba3e-825356c38116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:42.705402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b088c-f829-43a8-ba3e-825356c38116 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:42.705674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b088c-f829-43a8-ba3e-825356c38116 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:42.705934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b088c-f829-43a8-ba3e-825356c38116 None admin] PATH_INFO: `/v3/projects/a48d4d741809450a82825e526404656c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:42.707277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b088c-f829-43a8-ba3e-825356c38116 None admin] RBAC: Authorizing `identity:get_project(project_id=a48d4d741809450a82825e526404656c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:42.708754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b088c-f829-43a8-ba3e-825356c38116 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:42.710439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 569/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:42:42 2022] GET /identity/v3/projects/a48d4d741809450a82825e526404656c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:42.717270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314abae-ce3c-4ec5-8ffb-73103ed7cc8e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:42.834657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314abae-ce3c-4ec5-8ffb-73103ed7cc8e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:42.870261 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23743405-cf46-4fb4-972d-36e20badae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:42.871149 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23743405-cf46-4fb4-972d-36e20badae46 None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:42.871448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23743405-cf46-4fb4-972d-36e20badae46 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:42.871715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23743405-cf46-4fb4-972d-36e20badae46 None admin] PATH_INFO: `/v3/projects/a48d4d741809450a82825e526404656c/users/9f811f21c78343e3b16a4a859796bf0e/roles/d2054df5a6d24f5d90793b9b81cd4b4b` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:42.874326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23743405-cf46-4fb4-972d-36e20badae46 None admin] RBAC: Authorizing `identity:create_grant(project_id=a48d4d741809450a82825e526404656c, user_id=9f811f21c78343e3b16a4a859796bf0e, role_id=d2054df5a6d24f5d90793b9b81cd4b4b)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:42.875786 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23743405-cf46-4fb4-972d-36e20badae46 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:42.898063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 554/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:42:42 2022] PUT /identity/v3/projects/a48d4d741809450a82825e526404656c/users/9f811f21c78343e3b16a4a859796bf0e/roles/d2054df5a6d24f5d90793b9b81cd4b4b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:42:44.877343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b18b8e-aee9-42a8-afa1-1e66f7770793 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:44.877885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b18b8e-aee9-42a8-afa1-1e66f7770793 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:44.877885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b18b8e-aee9-42a8-afa1-1e66f7770793 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:44.879171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 570/1124] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:44.885823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bd440-8da8-41db-adab-2a998599b27f None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:44.886527 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bd440-8da8-41db-adab-2a998599b27f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:44.886696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bd440-8da8-41db-adab-2a998599b27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:44.897190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:44.915438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-063bd440-8da8-41db-adab-2a998599b27f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:44.916657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-063bd440-8da8-41db-adab-2a998599b27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:44.988086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 555/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:45.016327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b18b8e-aee9-42a8-afa1-1e66f7770793 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:45.062067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:45.128062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b18b8e-aee9-42a8-afa1-1e66f7770793 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:45.165000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2227346-a728-4293-b661-401afb94a61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:45.166279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2227346-a728-4293-b661-401afb94a61c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:45.166645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2227346-a728-4293-b661-401afb94a61c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:45.167004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2227346-a728-4293-b661-401afb94a61c None admin] PATH_INFO: `/v3/roles/d2054df5a6d24f5d90793b9b81cd4b4b` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:45.169665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2227346-a728-4293-b661-401afb94a61c None admin] RBAC: Authorizing `identity:get_role(role_id=d2054df5a6d24f5d90793b9b81cd4b4b)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:45.171908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2227346-a728-4293-b661-401afb94a61c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:45.173959 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 571/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:45 2022] GET /identity/v3/roles/d2054df5a6d24f5d90793b9b81cd4b4b => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:45.182412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bd440-8da8-41db-adab-2a998599b27f None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:45.283877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bd440-8da8-41db-adab-2a998599b27f None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:45.319429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76211-1061-498e-86fe-9b8e4b37bec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:45.320304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76211-1061-498e-86fe-9b8e4b37bec5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:45.320583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76211-1061-498e-86fe-9b8e4b37bec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:45.320879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76211-1061-498e-86fe-9b8e4b37bec5 None admin] PATH_INFO: `/v3/users/9f811f21c78343e3b16a4a859796bf0e` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:45.322076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b76211-1061-498e-86fe-9b8e4b37bec5 None admin] RBAC: Authorizing `identity:get_user(user_id=9f811f21c78343e3b16a4a859796bf0e)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:45.324251 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b76211-1061-498e-86fe-9b8e4b37bec5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:45.325887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 556/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:45 2022] GET /identity/v3/users/9f811f21c78343e3b16a4a859796bf0e => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:45.335240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2227346-a728-4293-b661-401afb94a61c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:45.451448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2227346-a728-4293-b661-401afb94a61c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:45.466906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:45.494543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f532a-2301-4a1c-a535-739cbf9c0fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:45.495168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f532a-2301-4a1c-a535-739cbf9c0fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:45.495499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f532a-2301-4a1c-a535-739cbf9c0fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:45.495774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f532a-2301-4a1c-a535-739cbf9c0fc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:45.577815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f532a-2301-4a1c-a535-739cbf9c0fc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:45.579292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f532a-2301-4a1c-a535-739cbf9c0fc9 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:45.679235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 572/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:42:45.688934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76211-1061-498e-86fe-9b8e4b37bec5 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:45.693793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:45.800935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76211-1061-498e-86fe-9b8e4b37bec5 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:45.837142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba82a73-17a1-463b-8e83-096214079856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:45.838004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba82a73-17a1-463b-8e83-096214079856 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:45.838283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba82a73-17a1-463b-8e83-096214079856 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:45.838553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba82a73-17a1-463b-8e83-096214079856 None admin] PATH_INFO: `/v3/projects/a48d4d741809450a82825e526404656c` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:45.839879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba82a73-17a1-463b-8e83-096214079856 None admin] RBAC: Authorizing `identity:get_project(project_id=a48d4d741809450a82825e526404656c)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:45.841328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba82a73-17a1-463b-8e83-096214079856 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:45.843037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 557/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:42:45 2022] GET /identity/v3/projects/a48d4d741809450a82825e526404656c => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:45.849396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f532a-2301-4a1c-a535-739cbf9c0fc9 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:45.861281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:45.960959 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f532a-2301-4a1c-a535-739cbf9c0fc9 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:45.996721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4ab2c-95e3-4d71-96fc-69cbcf25603b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:45.997379 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4ab2c-95e3-4d71-96fc-69cbcf25603b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:45.997659 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4ab2c-95e3-4d71-96fc-69cbcf25603b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:45.997931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4ab2c-95e3-4d71-96fc-69cbcf25603b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:45.998796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4ab2c-95e3-4d71-96fc-69cbcf25603b None admin] RBAC: Adding query filter params (role.id=d2054df5a6d24f5d90793b9b81cd4b4b, scope.project.id=a48d4d741809450a82825e526404656c, user.id=9f811f21c78343e3b16a4a859796bf0e) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:45.999105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4ab2c-95e3-4d71-96fc-69cbcf25603b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:46.000589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4ab2c-95e3-4d71-96fc-69cbcf25603b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:46.006586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 573/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:45 2022] GET /identity/v3/role_assignments?user.id=9f811f21c78343e3b16a4a859796bf0e&scope.project.id=a48d4d741809450a82825e526404656c&role.id=d2054df5a6d24f5d90793b9b81cd4b4b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:42:48.035067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65862d-afc8-4f62-b7e2-d3f7d194a1b7 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:48.035638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65862d-afc8-4f62-b7e2-d3f7d194a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:48.035749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65862d-afc8-4f62-b7e2-d3f7d194a1b7 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:48.037789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 558/1131] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:48.044084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d8c41-0902-49a6-9c80-50ac9a9e6d4b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:48.044472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d8c41-0902-49a6-9c80-50ac9a9e6d4b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:48.044633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d8c41-0902-49a6-9c80-50ac9a9e6d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:48.063167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6c3d8c41-0902-49a6-9c80-50ac9a9e6d4b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:48.064399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6c3d8c41-0902-49a6-9c80-50ac9a9e6d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:48.149829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 574/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:48.173944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65862d-afc8-4f62-b7e2-d3f7d194a1b7 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:48.267872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:48.300996 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65862d-afc8-4f62-b7e2-d3f7d194a1b7 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:48.338403 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c27ef-74c5-49b4-ad09-5ca0f1d28772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:48.339201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c27ef-74c5-49b4-ad09-5ca0f1d28772 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:48.339512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c27ef-74c5-49b4-ad09-5ca0f1d28772 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:48.339783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c27ef-74c5-49b4-ad09-5ca0f1d28772 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:48.340959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c27ef-74c5-49b4-ad09-5ca0f1d28772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:48.342392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c27ef-74c5-49b4-ad09-5ca0f1d28772 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:48.344053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 559/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:48.350752 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d8c41-0902-49a6-9c80-50ac9a9e6d4b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:48.355386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:48.461891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d8c41-0902-49a6-9c80-50ac9a9e6d4b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:48.502893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce309ebb-b29d-4d3f-8b9f-2747735d16fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:48.503611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce309ebb-b29d-4d3f-8b9f-2747735d16fa None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:48.503962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce309ebb-b29d-4d3f-8b9f-2747735d16fa None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:48.504239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce309ebb-b29d-4d3f-8b9f-2747735d16fa None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:48.504952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce309ebb-b29d-4d3f-8b9f-2747735d16fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:48.506403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce309ebb-b29d-4d3f-8b9f-2747735d16fa None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:48.541147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 575/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:42:48 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:50.560385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a864c-d2fa-40dc-bacf-ddc96743c65d None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:50.560891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a864c-d2fa-40dc-bacf-ddc96743c65d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:50.560891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a864c-d2fa-40dc-bacf-ddc96743c65d None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:50.562329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 560/1135] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:50.569120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ce2f4-fe03-47ac-9000-fc462718a4ad None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:50.569834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ce2f4-fe03-47ac-9000-fc462718a4ad None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:50.569980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ce2f4-fe03-47ac-9000-fc462718a4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:50.594827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-5b9ce2f4-fe03-47ac-9000-fc462718a4ad None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:50.596063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5b9ce2f4-fe03-47ac-9000-fc462718a4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:50.660031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:50.678677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 576/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:50.704092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a864c-d2fa-40dc-bacf-ddc96743c65d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:50.812149 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a864c-d2fa-40dc-bacf-ddc96743c65d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:50.868965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144ba34-675c-4018-a2e4-e5a5f3542a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:50.869654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144ba34-675c-4018-a2e4-e5a5f3542a6d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:50.869933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144ba34-675c-4018-a2e4-e5a5f3542a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:50.870211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144ba34-675c-4018-a2e4-e5a5f3542a6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:50.871624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144ba34-675c-4018-a2e4-e5a5f3542a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:50.873088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144ba34-675c-4018-a2e4-e5a5f3542a6d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:50.874751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 561/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:42:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:50.881258 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ce2f4-fe03-47ac-9000-fc462718a4ad None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:50.984924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ce2f4-fe03-47ac-9000-fc462718a4ad None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:51.022844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4b1c7-4619-4a9b-8f44-c74686d8a5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:51.023896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4b1c7-4619-4a9b-8f44-c74686d8a5ca None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:51.024462 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4b1c7-4619-4a9b-8f44-c74686d8a5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:51.024807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4b1c7-4619-4a9b-8f44-c74686d8a5ca None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:51.025918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4b1c7-4619-4a9b-8f44-c74686d8a5ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:51.028272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4b1c7-4619-4a9b-8f44-c74686d8a5ca None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:51.056705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 577/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:42:50 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:42:53.112935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf0414-66cb-4422-ad9d-b568911e2ddd None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:53.113497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf0414-66cb-4422-ad9d-b568911e2ddd None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:53.113497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf0414-66cb-4422-ad9d-b568911e2ddd None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:53.114864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 562/1139] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:53.121336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93266754-6273-4a80-8400-68de706d9d0c None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:53.121783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93266754-6273-4a80-8400-68de706d9d0c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:53.121911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93266754-6273-4a80-8400-68de706d9d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:53.144122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-93266754-6273-4a80-8400-68de706d9d0c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:53.145325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-93266754-6273-4a80-8400-68de706d9d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:53.181699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:53.222048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 578/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:53.245932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf0414-66cb-4422-ad9d-b568911e2ddd None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:53.255449 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:53.358188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf0414-66cb-4422-ad9d-b568911e2ddd None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:53.394707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a71955-f758-4af5-96a5-274fc76fc392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:53.395510 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a71955-f758-4af5-96a5-274fc76fc392 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:53.395704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a71955-f758-4af5-96a5-274fc76fc392 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:53.396601 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a71955-f758-4af5-96a5-274fc76fc392 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:53.406755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a71955-f758-4af5-96a5-274fc76fc392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-e1a71955-f758-4af5-96a5-274fc76fc392 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:53.414332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:53.415407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:53.416340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 563/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:42:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:53.422521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93266754-6273-4a80-8400-68de706d9d0c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:53.542341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93266754-6273-4a80-8400-68de706d9d0c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:53.578667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b64620-a9bb-4e30-bbb9-8ed0bb7aae3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:53.579363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b64620-a9bb-4e30-bbb9-8ed0bb7aae3a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:53.579639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b64620-a9bb-4e30-bbb9-8ed0bb7aae3a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:53.579909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b64620-a9bb-4e30-bbb9-8ed0bb7aae3a None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:53.580657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b64620-a9bb-4e30-bbb9-8ed0bb7aae3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:53.580966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b64620-a9bb-4e30-bbb9-8ed0bb7aae3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:53.582349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b64620-a9bb-4e30-bbb9-8ed0bb7aae3a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:53.590566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 579/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:42:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:53.596702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a71955-f758-4af5-96a5-274fc76fc392 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:53.666819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:53.703426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a71955-f758-4af5-96a5-274fc76fc392 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:53.739336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df129ae8-bf82-495d-8f56-8b04a15bf8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:53.740208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df129ae8-bf82-495d-8f56-8b04a15bf8bc None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:53.740488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df129ae8-bf82-495d-8f56-8b04a15bf8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:53.740766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df129ae8-bf82-495d-8f56-8b04a15bf8bc None admin] PATH_INFO: `/v3/users/4ba57d1bb986427d9c9c19ccd30f3c53` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:53.754366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df129ae8-bf82-495d-8f56-8b04a15bf8bc None admin] RBAC: Authorizing `identity:get_user(user_id=4ba57d1bb986427d9c9c19ccd30f3c53)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:53.755858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df129ae8-bf82-495d-8f56-8b04a15bf8bc None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:53.757519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 564/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:53 2022] GET /identity/v3/users/4ba57d1bb986427d9c9c19ccd30f3c53 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:53.764559 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b64620-a9bb-4e30-bbb9-8ed0bb7aae3a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:53.799521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:53.875668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b64620-a9bb-4e30-bbb9-8ed0bb7aae3a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:53.910804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18a5a1-ac1c-4b98-9e75-ab091ed4264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:53.911476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18a5a1-ac1c-4b98-9e75-ab091ed4264c None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:53.911752 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18a5a1-ac1c-4b98-9e75-ab091ed4264c None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:53.912020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18a5a1-ac1c-4b98-9e75-ab091ed4264c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:53.981466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18a5a1-ac1c-4b98-9e75-ab091ed4264c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:53.982910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18a5a1-ac1c-4b98-9e75-ab091ed4264c None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:54.015961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:54.082660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 580/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:42:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:42:54.092159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df129ae8-bf82-495d-8f56-8b04a15bf8bc None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:54.205756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df129ae8-bf82-495d-8f56-8b04a15bf8bc None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:54.229476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:54.245994 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fa0da-01ef-4737-a757-a1e89efbde26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:54.246793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fa0da-01ef-4737-a757-a1e89efbde26 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:54.247075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fa0da-01ef-4737-a757-a1e89efbde26 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:54.247374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fa0da-01ef-4737-a757-a1e89efbde26 None admin] PATH_INFO: `/v3/projects/91d029b103d2465fb52fa914339a0098` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:54.248598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fa0da-01ef-4737-a757-a1e89efbde26 None admin] RBAC: Authorizing `identity:get_project(project_id=91d029b103d2465fb52fa914339a0098)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:54.250035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fa0da-01ef-4737-a757-a1e89efbde26 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:54.251799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 565/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:42:54 2022] GET /identity/v3/projects/91d029b103d2465fb52fa914339a0098 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:54.257880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18a5a1-ac1c-4b98-9e75-ab091ed4264c None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:54.370891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18a5a1-ac1c-4b98-9e75-ab091ed4264c None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:54.407336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fffbd8-4f3f-4ef4-8be0-7780dff501c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:54.407972 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fffbd8-4f3f-4ef4-8be0-7780dff501c9 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:54.408245 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fffbd8-4f3f-4ef4-8be0-7780dff501c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:54.408514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fffbd8-4f3f-4ef4-8be0-7780dff501c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:54.409339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fffbd8-4f3f-4ef4-8be0-7780dff501c9 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=91d029b103d2465fb52fa914339a0098, user.id=4ba57d1bb986427d9c9c19ccd30f3c53) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:54.409641 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fffbd8-4f3f-4ef4-8be0-7780dff501c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:54.411068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fffbd8-4f3f-4ef4-8be0-7780dff501c9 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:54.412961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:54.419318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 581/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:42:54 2022] GET /identity/v3/role_assignments?user.id=4ba57d1bb986427d9c9c19ccd30f3c53&scope.project.id=91d029b103d2465fb52fa914339a0098&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:56.410634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f222b3f-635f-4bc3-a88e-53daead0d70c None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:56.411577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f222b3f-635f-4bc3-a88e-53daead0d70c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:56.411577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f222b3f-635f-4bc3-a88e-53daead0d70c None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:56.413811 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 566/1147] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:56.420180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46830563-1d13-42e7-9091-7622c732dcb3 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:56.420607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46830563-1d13-42e7-9091-7622c732dcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:56.420748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46830563-1d13-42e7-9091-7622c732dcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:56.442736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-46830563-1d13-42e7-9091-7622c732dcb3 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:56.444309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-46830563-1d13-42e7-9091-7622c732dcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:56.519119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 582/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:42:56.544385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f222b3f-635f-4bc3-a88e-53daead0d70c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:56.655428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f222b3f-635f-4bc3-a88e-53daead0d70c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:56.693271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8f12c-1cc3-472c-aa7e-9d210013d7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:56.693998 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8f12c-1cc3-472c-aa7e-9d210013d7ef None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:56.694276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8f12c-1cc3-472c-aa7e-9d210013d7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:56.694545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8f12c-1cc3-472c-aa7e-9d210013d7ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:56.705158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8f12c-1cc3-472c-aa7e-9d210013d7ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-36d8f12c-1cc3-472c-aa7e-9d210013d7ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:56.712924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:56.713924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:42:56.714868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 567/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:42:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:56.723074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46830563-1d13-42e7-9091-7622c732dcb3 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:56.830863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46830563-1d13-42e7-9091-7622c732dcb3 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:56.858905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:56.872852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e5dcd-ef5e-434e-9029-92c3f6b56bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:56.874012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e5dcd-ef5e-434e-9029-92c3f6b56bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:56.874521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e5dcd-ef5e-434e-9029-92c3f6b56bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:56.875016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e5dcd-ef5e-434e-9029-92c3f6b56bb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:56.876038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e5dcd-ef5e-434e-9029-92c3f6b56bb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:56.876545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e5dcd-ef5e-434e-9029-92c3f6b56bb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:56.878027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e5dcd-ef5e-434e-9029-92c3f6b56bb0 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:56.896825 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 583/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:42:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:56.905670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8f12c-1cc3-472c-aa7e-9d210013d7ef None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:56.962415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:57.018781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8f12c-1cc3-472c-aa7e-9d210013d7ef None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:57.061632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe0111-437b-4bee-9ef7-4823bc52dc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:57.062494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe0111-437b-4bee-9ef7-4823bc52dc7b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:57.062769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe0111-437b-4bee-9ef7-4823bc52dc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:57.063036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe0111-437b-4bee-9ef7-4823bc52dc7b None admin] PATH_INFO: `/v3/users/4ba57d1bb986427d9c9c19ccd30f3c53` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:57.064357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe0111-437b-4bee-9ef7-4823bc52dc7b None admin] RBAC: Authorizing `identity:get_user(user_id=4ba57d1bb986427d9c9c19ccd30f3c53)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:57.065822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe0111-437b-4bee-9ef7-4823bc52dc7b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:57.067496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 568/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:56 2022] GET /identity/v3/users/4ba57d1bb986427d9c9c19ccd30f3c53 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:57.075408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e5dcd-ef5e-434e-9029-92c3f6b56bb0 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:57.184410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e5dcd-ef5e-434e-9029-92c3f6b56bb0 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:57.218828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad71af-e282-40ee-8072-618026835957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:57.219632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad71af-e282-40ee-8072-618026835957 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:57.219919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad71af-e282-40ee-8072-618026835957 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:57.220191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad71af-e282-40ee-8072-618026835957 None admin] PATH_INFO: `/v3/projects/91d029b103d2465fb52fa914339a0098` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:57.221361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad71af-e282-40ee-8072-618026835957 None admin] RBAC: Authorizing `identity:get_project(project_id=91d029b103d2465fb52fa914339a0098)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:57.222804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad71af-e282-40ee-8072-618026835957 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:57.224550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 584/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:42:57 2022] GET /identity/v3/projects/91d029b103d2465fb52fa914339a0098 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:57.233739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe0111-437b-4bee-9ef7-4823bc52dc7b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:57.333793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe0111-437b-4bee-9ef7-4823bc52dc7b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:57.353726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:57.368674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47c0da-f724-4f06-b586-0824dbccffa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:57.369636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47c0da-f724-4f06-b586-0824dbccffa2 None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:57.369891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47c0da-f724-4f06-b586-0824dbccffa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:57.370157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47c0da-f724-4f06-b586-0824dbccffa2 None admin] PATH_INFO: `/v3/projects/91d029b103d2465fb52fa914339a0098/users/4ba57d1bb986427d9c9c19ccd30f3c53/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:57.372230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47c0da-f724-4f06-b586-0824dbccffa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=91d029b103d2465fb52fa914339a0098, user_id=4ba57d1bb986427d9c9c19ccd30f3c53, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:57.373666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47c0da-f724-4f06-b586-0824dbccffa2 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:57.389891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 569/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:42:57 2022] PUT /identity/v3/projects/91d029b103d2465fb52fa914339a0098/users/4ba57d1bb986427d9c9c19ccd30f3c53/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:42:59.401356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf604b1e-d9a4-4ffc-9c69-87c2afd2c653 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:59.401913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf604b1e-d9a4-4ffc-9c69-87c2afd2c653 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:59.401913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf604b1e-d9a4-4ffc-9c69-87c2afd2c653 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:59.404083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 585/1154] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:42:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:42:59.412106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b841e-74f5-4b1c-ad31-6ad4f4a07906 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:59.412570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b841e-74f5-4b1c-ad31-6ad4f4a07906 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:59.412670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b841e-74f5-4b1c-ad31-6ad4f4a07906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:59.436910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-1a2b841e-74f5-4b1c-ad31-6ad4f4a07906 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:42:59.438107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1a2b841e-74f5-4b1c-ad31-6ad4f4a07906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:42:59.505985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 570/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:42:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:42:59.533658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf604b1e-d9a4-4ffc-9c69-87c2afd2c653 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:59.596090 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:59.649739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf604b1e-d9a4-4ffc-9c69-87c2afd2c653 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:59.684446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c7f3f-73d7-497c-8b4d-4260d9142d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:59.685167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c7f3f-73d7-497c-8b4d-4260d9142d43 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:59.685445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c7f3f-73d7-497c-8b4d-4260d9142d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:59.685716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c7f3f-73d7-497c-8b4d-4260d9142d43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:59.697148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c7f3f-73d7-497c-8b4d-4260d9142d43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-489c7f3f-73d7-497c-8b4d-4260d9142d43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:42:59.704757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:42:59.705902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:42:59.707212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 586/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:42:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:42:59.712229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b841e-74f5-4b1c-ad31-6ad4f4a07906 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:59.817713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b841e-74f5-4b1c-ad31-6ad4f4a07906 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:59.828708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:42:59.857411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72fd5f-adff-455b-a422-f831094c0b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:42:59.858072 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72fd5f-adff-455b-a422-f831094c0b4f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:42:59.858349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72fd5f-adff-455b-a422-f831094c0b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:42:59.858616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72fd5f-adff-455b-a422-f831094c0b4f None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:42:59.859384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72fd5f-adff-455b-a422-f831094c0b4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:42:59.859686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72fd5f-adff-455b-a422-f831094c0b4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:42:59.861058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72fd5f-adff-455b-a422-f831094c0b4f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:42:59.869536 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 571/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:42:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:42:59.876194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c7f3f-73d7-497c-8b4d-4260d9142d43 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:42:59.980873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c7f3f-73d7-497c-8b4d-4260d9142d43 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:42:59.998242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:00.016760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d3539-3dbd-423b-b15d-5fbe92365175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:00.017614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d3539-3dbd-423b-b15d-5fbe92365175 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:00.017892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d3539-3dbd-423b-b15d-5fbe92365175 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:00.018152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d3539-3dbd-423b-b15d-5fbe92365175 None admin] PATH_INFO: `/v3/users/4ba57d1bb986427d9c9c19ccd30f3c53` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:00.019402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d3539-3dbd-423b-b15d-5fbe92365175 None admin] RBAC: Authorizing `identity:get_user(user_id=4ba57d1bb986427d9c9c19ccd30f3c53)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:00.020873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d3539-3dbd-423b-b15d-5fbe92365175 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:00.022623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 587/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:42:59 2022] GET /identity/v3/users/4ba57d1bb986427d9c9c19ccd30f3c53 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:00.028401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72fd5f-adff-455b-a422-f831094c0b4f None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:00.132259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72fd5f-adff-455b-a422-f831094c0b4f None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:00.166741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b10d8d-f5cf-4136-90a6-84cba9db37c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:00.167419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b10d8d-f5cf-4136-90a6-84cba9db37c3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:00.167696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b10d8d-f5cf-4136-90a6-84cba9db37c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:00.167968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b10d8d-f5cf-4136-90a6-84cba9db37c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:00.210230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:00.239486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b10d8d-f5cf-4136-90a6-84cba9db37c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:00.240895 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b10d8d-f5cf-4136-90a6-84cba9db37c3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:00.344412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 572/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:43:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:43:00.350687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d3539-3dbd-423b-b15d-5fbe92365175 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:00.468235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d3539-3dbd-423b-b15d-5fbe92365175 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:00.503740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c557e-e5c4-4f41-acca-468ac2684792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:00.504536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c557e-e5c4-4f41-acca-468ac2684792 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:00.504820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c557e-e5c4-4f41-acca-468ac2684792 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:00.505095 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c557e-e5c4-4f41-acca-468ac2684792 None admin] PATH_INFO: `/v3/projects/91d029b103d2465fb52fa914339a0098` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:00.506405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c557e-e5c4-4f41-acca-468ac2684792 None admin] RBAC: Authorizing `identity:get_project(project_id=91d029b103d2465fb52fa914339a0098)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:00.507910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c557e-e5c4-4f41-acca-468ac2684792 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:00.509657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 588/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:43:00 2022] GET /identity/v3/projects/91d029b103d2465fb52fa914339a0098 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:00.516140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b10d8d-f5cf-4136-90a6-84cba9db37c3 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:00.599705 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:00.631726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b10d8d-f5cf-4136-90a6-84cba9db37c3 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:00.671167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35609e5d-5032-480c-a8e8-f6ee39b7bd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:00.671949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35609e5d-5032-480c-a8e8-f6ee39b7bd31 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:00.672227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35609e5d-5032-480c-a8e8-f6ee39b7bd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:00.672497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35609e5d-5032-480c-a8e8-f6ee39b7bd31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:00.673334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35609e5d-5032-480c-a8e8-f6ee39b7bd31 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=91d029b103d2465fb52fa914339a0098, user.id=4ba57d1bb986427d9c9c19ccd30f3c53) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:43:00.673642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35609e5d-5032-480c-a8e8-f6ee39b7bd31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:00.675081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35609e5d-5032-480c-a8e8-f6ee39b7bd31 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:00.681113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 573/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:43:00 2022] GET /identity/v3/role_assignments?user.id=4ba57d1bb986427d9c9c19ccd30f3c53&scope.project.id=91d029b103d2465fb52fa914339a0098&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:02.710345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6603e9-a1cc-44ed-8acb-2b857af870c8 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:02.711297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6603e9-a1cc-44ed-8acb-2b857af870c8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:02.711297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6603e9-a1cc-44ed-8acb-2b857af870c8 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:02.713249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 589/1162] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:43:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:43:02.719683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a675e-2c55-43eb-af8c-a5b0f610d76c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:02.720091 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a675e-2c55-43eb-af8c-a5b0f610d76c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:02.720283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a675e-2c55-43eb-af8c-a5b0f610d76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:02.741099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4f1a675e-2c55-43eb-af8c-a5b0f610d76c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:43:02.742300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4f1a675e-2c55-43eb-af8c-a5b0f610d76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:43:02.816126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 574/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:43:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:43:02.839102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6603e9-a1cc-44ed-8acb-2b857af870c8 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:02.902058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:02.954690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6603e9-a1cc-44ed-8acb-2b857af870c8 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:02.992891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822e0c7-38b5-4a8b-afba-ef72254de211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:02.993537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822e0c7-38b5-4a8b-afba-ef72254de211 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:02.993843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822e0c7-38b5-4a8b-afba-ef72254de211 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:02.994119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822e0c7-38b5-4a8b-afba-ef72254de211 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:03.068625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822e0c7-38b5-4a8b-afba-ef72254de211 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:03.070010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822e0c7-38b5-4a8b-afba-ef72254de211 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:03.122871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:03.167932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 590/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:43:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:43:03.174236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a675e-2c55-43eb-af8c-a5b0f610d76c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:03.245002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:03.293886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a675e-2c55-43eb-af8c-a5b0f610d76c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:03.328295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ef0a7-941a-4f50-8acc-724b60676ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:03.329038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ef0a7-941a-4f50-8acc-724b60676ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:03.329320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ef0a7-941a-4f50-8acc-724b60676ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:03.329591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ef0a7-941a-4f50-8acc-724b60676ec0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:03.336328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ef0a7-941a-4f50-8acc-724b60676ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:03.337888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ef0a7-941a-4f50-8acc-724b60676ec0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-3e9ef0a7-941a-4f50-8acc-724b60676ec0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:43:03.352879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:43:03.354074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:43:03.355927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 575/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 4 05:43:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:43:03.363751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822e0c7-38b5-4a8b-afba-ef72254de211 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:03.475231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822e0c7-38b5-4a8b-afba-ef72254de211 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:03.513870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463a55d-b925-4195-b0fa-6587c0e8092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:03.514537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463a55d-b925-4195-b0fa-6587c0e8092e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:03.514815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463a55d-b925-4195-b0fa-6587c0e8092e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:03.515115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463a55d-b925-4195-b0fa-6587c0e8092e None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:03.515875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463a55d-b925-4195-b0fa-6587c0e8092e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:43:03.516550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463a55d-b925-4195-b0fa-6587c0e8092e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:03.517563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463a55d-b925-4195-b0fa-6587c0e8092e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:03.525118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 591/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 4 05:43:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:05.537252 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba3b7-4dee-4525-91ea-93b1d0304b0d None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:05.537697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba3b7-4dee-4525-91ea-93b1d0304b0d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:05.537788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba3b7-4dee-4525-91ea-93b1d0304b0d None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:05.539191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 576/1167] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:43:05.545534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a62d3-3576-42c9-b582-9085d223d4e5 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:05.545981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a62d3-3576-42c9-b582-9085d223d4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:05.546074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a62d3-3576-42c9-b582-9085d223d4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:05.568976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-0d7a62d3-3576-42c9-b582-9085d223d4e5 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:43:05.570184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0d7a62d3-3576-42c9-b582-9085d223d4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:43:05.584706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:05.644429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 592/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:43:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:43:05.668229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ba3b7-4dee-4525-91ea-93b1d0304b0d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:05.779494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ba3b7-4dee-4525-91ea-93b1d0304b0d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:05.805140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:05.815145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83ac33-406e-415c-8813-2a3dbca373cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:05.815850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83ac33-406e-415c-8813-2a3dbca373cc None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:05.816127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83ac33-406e-415c-8813-2a3dbca373cc None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:05.816404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83ac33-406e-415c-8813-2a3dbca373cc None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:05.817142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83ac33-406e-415c-8813-2a3dbca373cc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:05.818868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83ac33-406e-415c-8813-2a3dbca373cc None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:05.835556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 577/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Mon Jul 4 05:43:05 2022] POST /identity/v3/domains => generated 267 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:43:07.877306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dc4d2-6b21-4e87-bfe3-0050157a7b62 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:07.878143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dc4d2-6b21-4e87-bfe3-0050157a7b62 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:07.878143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dc4d2-6b21-4e87-bfe3-0050157a7b62 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:07.880383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 593/1170] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:43:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:43:07.885836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8b9da-d89d-486d-9d3c-52dbb3fe3965 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:07.886254 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8b9da-d89d-486d-9d3c-52dbb3fe3965 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:07.886342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8b9da-d89d-486d-9d3c-52dbb3fe3965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:07.905366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-f6c8b9da-d89d-486d-9d3c-52dbb3fe3965 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:43:07.906538 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f6c8b9da-d89d-486d-9d3c-52dbb3fe3965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:43:07.975202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 578/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:43:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:43:07.998621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dc4d2-6b21-4e87-bfe3-0050157a7b62 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:08.108924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dc4d2-6b21-4e87-bfe3-0050157a7b62 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:08.145483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be72e9d-5550-4e8a-8c4d-97a4b0a9f6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:08.146269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be72e9d-5550-4e8a-8c4d-97a4b0a9f6fe None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:08.146544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be72e9d-5550-4e8a-8c4d-97a4b0a9f6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:08.146814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be72e9d-5550-4e8a-8c4d-97a4b0a9f6fe None admin] PATH_INFO: `/v3/domains/16e27815e7214ccf8f12af88fab8586c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:08.157939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be72e9d-5550-4e8a-8c4d-97a4b0a9f6fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=16e27815e7214ccf8f12af88fab8586c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:08.159446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be72e9d-5550-4e8a-8c4d-97a4b0a9f6fe None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:08.161063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 594/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Jul 4 05:43:07 2022] GET /identity/v3/domains/16e27815e7214ccf8f12af88fab8586c => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:08.167078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8b9da-d89d-486d-9d3c-52dbb3fe3965 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:08.272367 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8b9da-d89d-486d-9d3c-52dbb3fe3965 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:08.291194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:08.311757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eab0e4-4a87-401d-a7ae-638f01ac1266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:08.312420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eab0e4-4a87-401d-a7ae-638f01ac1266 None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:08.312702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eab0e4-4a87-401d-a7ae-638f01ac1266 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:08.312971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eab0e4-4a87-401d-a7ae-638f01ac1266 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:08.313691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eab0e4-4a87-401d-a7ae-638f01ac1266 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:08.315138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eab0e4-4a87-401d-a7ae-638f01ac1266 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:08.343704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 579/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:43:08 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:43:10.380515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5668a-f9e3-48ab-bbf0-f058ea8ac7a8 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:10.381407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5668a-f9e3-48ab-bbf0-f058ea8ac7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:10.381407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5668a-f9e3-48ab-bbf0-f058ea8ac7a8 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:10.383563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 595/1174] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:43:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:43:10.389070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05ebd2-1e68-42ca-9dbd-abac41590812 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:10.389511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05ebd2-1e68-42ca-9dbd-abac41590812 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:10.389611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05ebd2-1e68-42ca-9dbd-abac41590812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:10.407719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-fc05ebd2-1e68-42ca-9dbd-abac41590812 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:43:10.408927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fc05ebd2-1e68-42ca-9dbd-abac41590812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:43:10.483012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 580/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:43:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:43:10.511560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5668a-f9e3-48ab-bbf0-f058ea8ac7a8 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:10.524393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:10.619580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5668a-f9e3-48ab-bbf0-f058ea8ac7a8 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:10.653486 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bc5ae-18fd-4750-a80a-72f6cb49ec03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:10.654284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bc5ae-18fd-4750-a80a-72f6cb49ec03 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:10.654563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bc5ae-18fd-4750-a80a-72f6cb49ec03 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:10.654834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bc5ae-18fd-4750-a80a-72f6cb49ec03 None admin] PATH_INFO: `/v3/domains/16e27815e7214ccf8f12af88fab8586c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:10.656056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bc5ae-18fd-4750-a80a-72f6cb49ec03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=16e27815e7214ccf8f12af88fab8586c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:10.657510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bc5ae-18fd-4750-a80a-72f6cb49ec03 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:10.659154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 596/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Jul 4 05:43:10 2022] GET /identity/v3/domains/16e27815e7214ccf8f12af88fab8586c => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:10.665669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05ebd2-1e68-42ca-9dbd-abac41590812 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:10.770129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:10.774132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05ebd2-1e68-42ca-9dbd-abac41590812 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:10.809927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a9425-1a7e-4c4c-a1e9-46fc1fc24cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:10.810595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a9425-1a7e-4c4c-a1e9-46fc1fc24cea None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:10.810874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a9425-1a7e-4c4c-a1e9-46fc1fc24cea None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:10.811126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a9425-1a7e-4c4c-a1e9-46fc1fc24cea None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:10.811902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a9425-1a7e-4c4c-a1e9-46fc1fc24cea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:10.813551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a9425-1a7e-4c4c-a1e9-46fc1fc24cea None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:10.840678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 581/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:43:10 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:43:12.904344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddfc68-7117-4f48-b5be-0f82446d17bb None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:12.905161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddfc68-7117-4f48-b5be-0f82446d17bb None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:12.905161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddfc68-7117-4f48-b5be-0f82446d17bb None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:12.907304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 597/1178] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:43:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:43:12.912799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c647-ba95-4275-b053-66d0a05b27c7 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:12.913221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c647-ba95-4275-b053-66d0a05b27c7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:12.913330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c647-ba95-4275-b053-66d0a05b27c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:12.931788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-1a28c647-ba95-4275-b053-66d0a05b27c7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:43:12.932967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1a28c647-ba95-4275-b053-66d0a05b27c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:43:13.003571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 582/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:43:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:43:13.031311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddfc68-7117-4f48-b5be-0f82446d17bb None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:13.125395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:13.154583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddfc68-7117-4f48-b5be-0f82446d17bb None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:13.189099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad15a64-611f-4d24-b9bb-e4a8b0aa3c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:13.189882 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad15a64-611f-4d24-b9bb-e4a8b0aa3c29 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:13.190153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad15a64-611f-4d24-b9bb-e4a8b0aa3c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:13.190414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad15a64-611f-4d24-b9bb-e4a8b0aa3c29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:13.202218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad15a64-611f-4d24-b9bb-e4a8b0aa3c29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-8ad15a64-611f-4d24-b9bb-e4a8b0aa3c29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:43:13.209743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:43:13.210733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:43:13.211670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 598/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:43:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:43:13.219993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28c647-ba95-4275-b053-66d0a05b27c7 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:13.307696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:13.330000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28c647-ba95-4275-b053-66d0a05b27c7 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:13.367041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985497f-51ce-46d6-9829-5f707110462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:13.367771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985497f-51ce-46d6-9829-5f707110462f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:13.368130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985497f-51ce-46d6-9829-5f707110462f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:13.368429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985497f-51ce-46d6-9829-5f707110462f None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:13.369166 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985497f-51ce-46d6-9829-5f707110462f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:43:13.369466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985497f-51ce-46d6-9829-5f707110462f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:13.370861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985497f-51ce-46d6-9829-5f707110462f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:13.380113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 583/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:43:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:13.386900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad15a64-611f-4d24-b9bb-e4a8b0aa3c29 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:13.514143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad15a64-611f-4d24-b9bb-e4a8b0aa3c29 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:13.552402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478c1ba-1c2f-4706-bb64-0ddc4a8286b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:13.553293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478c1ba-1c2f-4706-bb64-0ddc4a8286b3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:13.553573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478c1ba-1c2f-4706-bb64-0ddc4a8286b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:13.553845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478c1ba-1c2f-4706-bb64-0ddc4a8286b3 None admin] PATH_INFO: `/v3/users/05d9274d9b3e48b2932250751e114736` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:13.556688 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:13.568208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478c1ba-1c2f-4706-bb64-0ddc4a8286b3 None admin] RBAC: Authorizing `identity:get_user(user_id=05d9274d9b3e48b2932250751e114736)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:13.569671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478c1ba-1c2f-4706-bb64-0ddc4a8286b3 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:13.571343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 599/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:43:13 2022] GET /identity/v3/users/05d9274d9b3e48b2932250751e114736 => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:13.580675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985497f-51ce-46d6-9829-5f707110462f None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:13.686642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985497f-51ce-46d6-9829-5f707110462f None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:13.722526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c0fc6-b8fb-4be7-9f81-a35473c2ff05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:13.723537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c0fc6-b8fb-4be7-9f81-a35473c2ff05 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:13.723903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c0fc6-b8fb-4be7-9f81-a35473c2ff05 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:13.724240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c0fc6-b8fb-4be7-9f81-a35473c2ff05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:13.743342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:13.799914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c0fc6-b8fb-4be7-9f81-a35473c2ff05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:13.801313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c0fc6-b8fb-4be7-9f81-a35473c2ff05 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:13.905535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 584/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:43:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:43:13.914437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478c1ba-1c2f-4706-bb64-0ddc4a8286b3 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:14.020761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478c1ba-1c2f-4706-bb64-0ddc4a8286b3 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:14.055601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48db8d1-de1d-4040-ac07-4110e4258ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:14.056379 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48db8d1-de1d-4040-ac07-4110e4258ebb None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:14.056752 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48db8d1-de1d-4040-ac07-4110e4258ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:14.057021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48db8d1-de1d-4040-ac07-4110e4258ebb None admin] PATH_INFO: `/v3/projects/b3fa8c9a137e40a79fcdaff1e995a613` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:14.058307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48db8d1-de1d-4040-ac07-4110e4258ebb None admin] RBAC: Authorizing `identity:get_project(project_id=b3fa8c9a137e40a79fcdaff1e995a613)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:14.059787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48db8d1-de1d-4040-ac07-4110e4258ebb None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:14.061451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 600/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:43:13 2022] GET /identity/v3/projects/b3fa8c9a137e40a79fcdaff1e995a613 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:43:14.068075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c0fc6-b8fb-4be7-9f81-a35473c2ff05 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:14.116272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:14.168922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c0fc6-b8fb-4be7-9f81-a35473c2ff05 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:14.205463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22996f17-950f-449a-871a-0ee95f9fcee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:14.206104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22996f17-950f-449a-871a-0ee95f9fcee5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:14.206383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22996f17-950f-449a-871a-0ee95f9fcee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:14.206656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22996f17-950f-449a-871a-0ee95f9fcee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:14.207591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22996f17-950f-449a-871a-0ee95f9fcee5 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=b3fa8c9a137e40a79fcdaff1e995a613, user.id=05d9274d9b3e48b2932250751e114736) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:43:14.207898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22996f17-950f-449a-871a-0ee95f9fcee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:14.209325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22996f17-950f-449a-871a-0ee95f9fcee5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:14.214785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 585/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:43:14 2022] GET /identity/v3/role_assignments?user.id=05d9274d9b3e48b2932250751e114736&scope.project.id=b3fa8c9a137e40a79fcdaff1e995a613&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:16.178491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec71a8a-0a02-4974-b33c-4f8b166b5508 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:16.179389 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec71a8a-0a02-4974-b33c-4f8b166b5508 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:16.179389 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec71a8a-0a02-4974-b33c-4f8b166b5508 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:16.181691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 601/1186] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:43:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:43:16.187390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2496dab-2a23-488d-9d99-a80781da1f24 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:16.187846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2496dab-2a23-488d-9d99-a80781da1f24 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:16.187943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2496dab-2a23-488d-9d99-a80781da1f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:16.206571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a2496dab-2a23-488d-9d99-a80781da1f24 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:43:16.207765 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a2496dab-2a23-488d-9d99-a80781da1f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:43:16.276340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 586/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:43:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:43:16.299406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec71a8a-0a02-4974-b33c-4f8b166b5508 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:16.372647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:16.417239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec71a8a-0a02-4974-b33c-4f8b166b5508 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:16.455073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19f7eb-eb6a-42f3-8dd7-25b8c095a649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:16.455820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19f7eb-eb6a-42f3-8dd7-25b8c095a649 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:16.456098 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19f7eb-eb6a-42f3-8dd7-25b8c095a649 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:16.456374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19f7eb-eb6a-42f3-8dd7-25b8c095a649 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:16.466578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19f7eb-eb6a-42f3-8dd7-25b8c095a649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-4c19f7eb-eb6a-42f3-8dd7-25b8c095a649 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:43:16.474158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:43:16.475300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:43:16.476413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 602/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:43:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:43:16.483168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2496dab-2a23-488d-9d99-a80781da1f24 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:16.536483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:16.608911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2496dab-2a23-488d-9d99-a80781da1f24 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:16.645156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18937d-11ca-4e75-8583-6c7bd02e5481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:16.645854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18937d-11ca-4e75-8583-6c7bd02e5481 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:16.646133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18937d-11ca-4e75-8583-6c7bd02e5481 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:16.646415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18937d-11ca-4e75-8583-6c7bd02e5481 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:16.647161 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18937d-11ca-4e75-8583-6c7bd02e5481 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:43:16.647498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18937d-11ca-4e75-8583-6c7bd02e5481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:16.648878 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18937d-11ca-4e75-8583-6c7bd02e5481 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:16.657032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 587/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:43:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:43:16.663386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19f7eb-eb6a-42f3-8dd7-25b8c095a649 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:16.764701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19f7eb-eb6a-42f3-8dd7-25b8c095a649 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:16.788454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:16.801001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d348ae-71dc-4bf7-8a8c-87a8398ad9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:16.801947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d348ae-71dc-4bf7-8a8c-87a8398ad9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:16.802230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d348ae-71dc-4bf7-8a8c-87a8398ad9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:16.802498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d348ae-71dc-4bf7-8a8c-87a8398ad9a4 None admin] PATH_INFO: `/v3/users/05d9274d9b3e48b2932250751e114736` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:16.803705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d348ae-71dc-4bf7-8a8c-87a8398ad9a4 None admin] RBAC: Authorizing `identity:get_user(user_id=05d9274d9b3e48b2932250751e114736)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:16.805135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d348ae-71dc-4bf7-8a8c-87a8398ad9a4 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:16.806811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 603/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:43:16 2022] GET /identity/v3/users/05d9274d9b3e48b2932250751e114736 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:16.816277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18937d-11ca-4e75-8583-6c7bd02e5481 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:16.928685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18937d-11ca-4e75-8583-6c7bd02e5481 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:16.930259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:16.965431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a93f4a-b893-4e89-891d-50d353915a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:16.966286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a93f4a-b893-4e89-891d-50d353915a8d None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:16.966572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a93f4a-b893-4e89-891d-50d353915a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:16.966838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a93f4a-b893-4e89-891d-50d353915a8d None admin] PATH_INFO: `/v3/projects/b3fa8c9a137e40a79fcdaff1e995a613` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:16.968048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a93f4a-b893-4e89-891d-50d353915a8d None admin] RBAC: Authorizing `identity:get_project(project_id=b3fa8c9a137e40a79fcdaff1e995a613)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:16.969495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a93f4a-b893-4e89-891d-50d353915a8d None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:16.971194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 588/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:43:16 2022] GET /identity/v3/projects/b3fa8c9a137e40a79fcdaff1e995a613 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:16.977570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d348ae-71dc-4bf7-8a8c-87a8398ad9a4 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:17.103655 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d348ae-71dc-4bf7-8a8c-87a8398ad9a4 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:17.140108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5875b2-23df-48d3-9c5c-3e0069c7bf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:17.140997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5875b2-23df-48d3-9c5c-3e0069c7bf78 None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:17.141278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5875b2-23df-48d3-9c5c-3e0069c7bf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:17.141550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5875b2-23df-48d3-9c5c-3e0069c7bf78 None admin] PATH_INFO: `/v3/projects/b3fa8c9a137e40a79fcdaff1e995a613/users/05d9274d9b3e48b2932250751e114736/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:17.143593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5875b2-23df-48d3-9c5c-3e0069c7bf78 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3fa8c9a137e40a79fcdaff1e995a613, user_id=05d9274d9b3e48b2932250751e114736, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:17.145046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5875b2-23df-48d3-9c5c-3e0069c7bf78 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:17.161165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 604/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:43:16 2022] PUT /identity/v3/projects/b3fa8c9a137e40a79fcdaff1e995a613/users/05d9274d9b3e48b2932250751e114736/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:43:19.182663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235401b-0188-402c-a93c-f267dcbb83e0 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:19.183296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235401b-0188-402c-a93c-f267dcbb83e0 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:19.183296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235401b-0188-402c-a93c-f267dcbb83e0 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:19.184617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 589/1193] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:43:19.190912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64faa61-22c7-4316-b55b-bd05fbf19a25 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:19.191693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64faa61-22c7-4316-b55b-bd05fbf19a25 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:19.191830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64faa61-22c7-4316-b55b-bd05fbf19a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:19.217359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d64faa61-22c7-4316-b55b-bd05fbf19a25 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:43:19.218554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d64faa61-22c7-4316-b55b-bd05fbf19a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:43:19.266803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:19.295956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 605/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:43:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:43:19.320074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235401b-0188-402c-a93c-f267dcbb83e0 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:19.430278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235401b-0188-402c-a93c-f267dcbb83e0 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:19.465292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d9cd7-45f7-4539-8795-fc4b71bcc9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:19.466006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d9cd7-45f7-4539-8795-fc4b71bcc9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:19.466282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d9cd7-45f7-4539-8795-fc4b71bcc9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:19.466548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d9cd7-45f7-4539-8795-fc4b71bcc9a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:19.476588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d9cd7-45f7-4539-8795-fc4b71bcc9a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-652d9cd7-45f7-4539-8795-fc4b71bcc9a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:43:19.484340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:43:19.485378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:43:19.486263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 590/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:43:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:43:19.493965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64faa61-22c7-4316-b55b-bd05fbf19a25 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:19.612070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64faa61-22c7-4316-b55b-bd05fbf19a25 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:19.651680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da922e97-05fa-4b4f-8891-0536be254c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:19.652338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da922e97-05fa-4b4f-8891-0536be254c40 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:19.652612 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da922e97-05fa-4b4f-8891-0536be254c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:19.652883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da922e97-05fa-4b4f-8891-0536be254c40 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:19.653631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da922e97-05fa-4b4f-8891-0536be254c40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:43:19.653936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da922e97-05fa-4b4f-8891-0536be254c40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:19.655335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da922e97-05fa-4b4f-8891-0536be254c40 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:19.663637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 606/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:43:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:43:19.672448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d9cd7-45f7-4539-8795-fc4b71bcc9a4 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:19.708716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:19.779880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d9cd7-45f7-4539-8795-fc4b71bcc9a4 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:19.814342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa87882-4da5-4363-a7da-3479162fe736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:19.815212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa87882-4da5-4363-a7da-3479162fe736 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:19.815523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa87882-4da5-4363-a7da-3479162fe736 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:19.815791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa87882-4da5-4363-a7da-3479162fe736 None admin] PATH_INFO: `/v3/users/05d9274d9b3e48b2932250751e114736` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:19.817056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa87882-4da5-4363-a7da-3479162fe736 None admin] RBAC: Authorizing `identity:get_user(user_id=05d9274d9b3e48b2932250751e114736)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:19.818859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa87882-4da5-4363-a7da-3479162fe736 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:19.820529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 591/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 4 05:43:19 2022] GET /identity/v3/users/05d9274d9b3e48b2932250751e114736 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:19.827310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da922e97-05fa-4b4f-8891-0536be254c40 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:19.877948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:19.949065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da922e97-05fa-4b4f-8891-0536be254c40 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:19.985614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef3b20-f9f5-4a24-9304-7553cb505081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:19.986251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef3b20-f9f5-4a24-9304-7553cb505081 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:19.986529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef3b20-f9f5-4a24-9304-7553cb505081 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:19.986803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef3b20-f9f5-4a24-9304-7553cb505081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:20.059494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef3b20-f9f5-4a24-9304-7553cb505081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:20.060874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef3b20-f9f5-4a24-9304-7553cb505081 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:20.103794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:20.162779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 607/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:43:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:43:20.169097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa87882-4da5-4363-a7da-3479162fe736 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:20.272252 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:20.276378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa87882-4da5-4363-a7da-3479162fe736 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:20.316104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49f621-3cf7-44fa-b7b5-87b9ee86c11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:20.316890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f621-3cf7-44fa-b7b5-87b9ee86c11c None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:20.317167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f621-3cf7-44fa-b7b5-87b9ee86c11c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:20.317779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f621-3cf7-44fa-b7b5-87b9ee86c11c None admin] PATH_INFO: `/v3/projects/b3fa8c9a137e40a79fcdaff1e995a613` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:20.319059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49f621-3cf7-44fa-b7b5-87b9ee86c11c None admin] RBAC: Authorizing `identity:get_project(project_id=b3fa8c9a137e40a79fcdaff1e995a613)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:20.320539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49f621-3cf7-44fa-b7b5-87b9ee86c11c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:20.322367 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 592/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 4 05:43:20 2022] GET /identity/v3/projects/b3fa8c9a137e40a79fcdaff1e995a613 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:20.328814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef3b20-f9f5-4a24-9304-7553cb505081 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:43:20.438827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef3b20-f9f5-4a24-9304-7553cb505081 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:43:20.474056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24336ac-5e6d-45cc-9d0b-cee8c8d5392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:43:20.474695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24336ac-5e6d-45cc-9d0b-cee8c8d5392e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:43:20.474970 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24336ac-5e6d-45cc-9d0b-cee8c8d5392e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:43:20.475221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24336ac-5e6d-45cc-9d0b-cee8c8d5392e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:43:20.476085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24336ac-5e6d-45cc-9d0b-cee8c8d5392e None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=b3fa8c9a137e40a79fcdaff1e995a613, user.id=05d9274d9b3e48b2932250751e114736) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:43:20.476390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24336ac-5e6d-45cc-9d0b-cee8c8d5392e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:43:20.477831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24336ac-5e6d-45cc-9d0b-cee8c8d5392e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:43:20.483750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 608/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:43:20 2022] GET /identity/v3/role_assignments?user.id=05d9274d9b3e48b2932250751e114736&scope.project.id=b3fa8c9a137e40a79fcdaff1e995a613&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:43:30.310221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=17 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:43:30.481510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:16.159365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5da10-d1ca-48b0-8a10-86ed1ffdb8a8 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:16.159974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5da10-d1ca-48b0-8a10-86ed1ffdb8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:16.159974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5da10-d1ca-48b0-8a10-86ed1ffdb8a8 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:16.161296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 593/1201] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:16.167111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95759d8-2e36-44b6-af34-67f2b07acfd0 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:16.167682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95759d8-2e36-44b6-af34-67f2b07acfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:16.167785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95759d8-2e36-44b6-af34-67f2b07acfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:16.186498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b95759d8-2e36-44b6-af34-67f2b07acfd0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:16.187986 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b95759d8-2e36-44b6-af34-67f2b07acfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:16.276833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 609/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:44:16.303393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5da10-d1ca-48b0-8a10-86ed1ffdb8a8 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:16.435178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5da10-d1ca-48b0-8a10-86ed1ffdb8a8 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:16.483614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaafe8a-83b4-4696-8609-410621de5b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:16.484616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaafe8a-83b4-4696-8609-410621de5b51 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:16.484850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaafe8a-83b4-4696-8609-410621de5b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:16.485152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaafe8a-83b4-4696-8609-410621de5b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:16.496390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaafe8a-83b4-4696-8609-410621de5b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:16.497900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaafe8a-83b4-4696-8609-410621de5b51 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-cdaafe8a-83b4-4696-8609-410621de5b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:16.526344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:16.527572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:16.528550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:16.528550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:16.528550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:16.528550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:16.528550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 594/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:16.533806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95759d8-2e36-44b6-af34-67f2b07acfd0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:16.659930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95759d8-2e36-44b6-af34-67f2b07acfd0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:16.695333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:16.703510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e7e4e-6a29-4218-8a4b-ede9986ae4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:16.704197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e7e4e-6a29-4218-8a4b-ede9986ae4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:16.704481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e7e4e-6a29-4218-8a4b-ede9986ae4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:16.704753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e7e4e-6a29-4218-8a4b-ede9986ae4a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:16.705483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e7e4e-6a29-4218-8a4b-ede9986ae4a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:16.705809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e7e4e-6a29-4218-8a4b-ede9986ae4a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:16.708096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e7e4e-6a29-4218-8a4b-ede9986ae4a9 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:16.722398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 610/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:16.728353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaafe8a-83b4-4696-8609-410621de5b51 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:16.793707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:16.849604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaafe8a-83b4-4696-8609-410621de5b51 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:16.895137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94ea07-d71e-4b9d-b322-8881ef8f8aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:16.896182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94ea07-d71e-4b9d-b322-8881ef8f8aca None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:16.896467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94ea07-d71e-4b9d-b322-8881ef8f8aca None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:16.896786 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94ea07-d71e-4b9d-b322-8881ef8f8aca None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:16.897503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94ea07-d71e-4b9d-b322-8881ef8f8aca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:16.899068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94ea07-d71e-4b9d-b322-8881ef8f8aca None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:16.924340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 595/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 4 05:44:16 2022] POST /identity/v3/users => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:44:19.203435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7894d7-a1ed-40ca-a7ee-29acd4226a47 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:19.204058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7894d7-a1ed-40ca-a7ee-29acd4226a47 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:19.204058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7894d7-a1ed-40ca-a7ee-29acd4226a47 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:19.207279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 611/1206] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:19.211125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39845748-6341-4176-b7c1-a2d2e27e3bd9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:19.211569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39845748-6341-4176-b7c1-a2d2e27e3bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:19.211774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39845748-6341-4176-b7c1-a2d2e27e3bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:19.229896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-39845748-6341-4176-b7c1-a2d2e27e3bd9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:19.231127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-39845748-6341-4176-b7c1-a2d2e27e3bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:19.313003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 596/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:44:19.338438 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7894d7-a1ed-40ca-a7ee-29acd4226a47 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:19.471879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7894d7-a1ed-40ca-a7ee-29acd4226a47 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:19.521992 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5be6b-a72b-4a19-b193-93d4d78bb9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:19.522932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5be6b-a72b-4a19-b193-93d4d78bb9cf None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:19.523205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5be6b-a72b-4a19-b193-93d4d78bb9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:19.523787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5be6b-a72b-4a19-b193-93d4d78bb9cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:19.538275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5be6b-a72b-4a19-b193-93d4d78bb9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-74f5be6b-a72b-4a19-b193-93d4d78bb9cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:44:19.547741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:19.548787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:19.549650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 612/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:44:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:19.555684 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39845748-6341-4176-b7c1-a2d2e27e3bd9 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:19.603789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:19.673341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39845748-6341-4176-b7c1-a2d2e27e3bd9 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:19.709567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc280f-3032-40a2-942e-77adf24c129a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:19.710239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc280f-3032-40a2-942e-77adf24c129a None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:19.710526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc280f-3032-40a2-942e-77adf24c129a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:19.710790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc280f-3032-40a2-942e-77adf24c129a None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:19.711743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc280f-3032-40a2-942e-77adf24c129a None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:19.712048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc280f-3032-40a2-942e-77adf24c129a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:19.713489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc280f-3032-40a2-942e-77adf24c129a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:19.728914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 597/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:44:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:19.731099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5be6b-a72b-4a19-b193-93d4d78bb9cf None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:19.784652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:19.838769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5be6b-a72b-4a19-b193-93d4d78bb9cf None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:19.879849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f5a98-6cb9-4ea3-bf40-6965d2f1dd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:19.881090 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f5a98-6cb9-4ea3-bf40-6965d2f1dd27 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:19.881439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f5a98-6cb9-4ea3-bf40-6965d2f1dd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:19.881841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f5a98-6cb9-4ea3-bf40-6965d2f1dd27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:19.893469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f5a98-6cb9-4ea3-bf40-6965d2f1dd27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:19.894969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f5a98-6cb9-4ea3-bf40-6965d2f1dd27 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-cf2f5a98-6cb9-4ea3-bf40-6965d2f1dd27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:19.913294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:19.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:19.920289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:19.920289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:19.920289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:19.920289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:19.920289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 613/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:44:19.920584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc280f-3032-40a2-942e-77adf24c129a None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:20.029041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:20.032981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc280f-3032-40a2-942e-77adf24c129a None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:20.079029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b94a0-102e-472f-9083-64a71218ae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:20.079784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b94a0-102e-472f-9083-64a71218ae04 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:20.080112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b94a0-102e-472f-9083-64a71218ae04 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:20.080509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b94a0-102e-472f-9083-64a71218ae04 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:20.081724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b94a0-102e-472f-9083-64a71218ae04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:20.082063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b94a0-102e-472f-9083-64a71218ae04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:20.083717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b94a0-102e-472f-9083-64a71218ae04 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:20.106795 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 598/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:44:20.108928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f5a98-6cb9-4ea3-bf40-6965d2f1dd27 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:20.230870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f5a98-6cb9-4ea3-bf40-6965d2f1dd27 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:20.232808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:20.271510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500be093-8be1-41b6-90b7-ecce5b5b87c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:20.272333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500be093-8be1-41b6-90b7-ecce5b5b87c9 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:20.272610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500be093-8be1-41b6-90b7-ecce5b5b87c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:20.272878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500be093-8be1-41b6-90b7-ecce5b5b87c9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:20.283353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500be093-8be1-41b6-90b7-ecce5b5b87c9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:20.283353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500be093-8be1-41b6-90b7-ecce5b5b87c9 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-500be093-8be1-41b6-90b7-ecce5b5b87c9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:44:20.297614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:20.298677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:20.299709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 614/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:20.304807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b94a0-102e-472f-9083-64a71218ae04 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:20.414795 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b94a0-102e-472f-9083-64a71218ae04 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:20.455272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2179d-4201-45f1-acce-d206427225f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:20.456134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2179d-4201-45f1-acce-d206427225f7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:20.456425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2179d-4201-45f1-acce-d206427225f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:20.456776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2179d-4201-45f1-acce-d206427225f7 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:20.457774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2179d-4201-45f1-acce-d206427225f7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:20.458129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2179d-4201-45f1-acce-d206427225f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:20.459621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2179d-4201-45f1-acce-d206427225f7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:20.485074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 599/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 4 05:44:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:20.486736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500be093-8be1-41b6-90b7-ecce5b5b87c9 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:20.611751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500be093-8be1-41b6-90b7-ecce5b5b87c9 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:20.655952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226c728-4a3c-4f54-a6ee-02158151c486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:20.656605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226c728-4a3c-4f54-a6ee-02158151c486 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:20.656892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226c728-4a3c-4f54-a6ee-02158151c486 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:20.657131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226c728-4a3c-4f54-a6ee-02158151c486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:20.687990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:20.728030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226c728-4a3c-4f54-a6ee-02158151c486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:20.731122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226c728-4a3c-4f54-a6ee-02158151c486 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:20.840763 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 615/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:44:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:44:20.846550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2179d-4201-45f1-acce-d206427225f7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:20.866661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:20.971789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2179d-4201-45f1-acce-d206427225f7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:21.017254 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f8127-698c-4b3c-972e-01accf2feb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:21.018489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f8127-698c-4b3c-972e-01accf2feb5f None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:21.018965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f8127-698c-4b3c-972e-01accf2feb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:21.019493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f8127-698c-4b3c-972e-01accf2feb5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:21.031091 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f8127-698c-4b3c-972e-01accf2feb5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:21.032635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f8127-698c-4b3c-972e-01accf2feb5f None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-f36f8127-698c-4b3c-972e-01accf2feb5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:21.053509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:21.054514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:21.055477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:21.055477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:21.055477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:21.055477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:21.055477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 600/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:21.061411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226c728-4a3c-4f54-a6ee-02158151c486 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:21.133425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:21.178214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226c728-4a3c-4f54-a6ee-02158151c486 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:21.213415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db2b99-c82a-4022-81ee-2c34af5eed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:21.214137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db2b99-c82a-4022-81ee-2c34af5eed27 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:21.214426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db2b99-c82a-4022-81ee-2c34af5eed27 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:21.214701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db2b99-c82a-4022-81ee-2c34af5eed27 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:21.215450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db2b99-c82a-4022-81ee-2c34af5eed27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:21.215754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db2b99-c82a-4022-81ee-2c34af5eed27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:21.217146 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db2b99-c82a-4022-81ee-2c34af5eed27 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:21.229084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 616/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:21.235802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f8127-698c-4b3c-972e-01accf2feb5f None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:21.322271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:21.362112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f8127-698c-4b3c-972e-01accf2feb5f None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:21.401097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a216f4f-5f3b-405d-98cc-fd98e8b7601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:21.402283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a216f4f-5f3b-405d-98cc-fd98e8b7601e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:21.402560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a216f4f-5f3b-405d-98cc-fd98e8b7601e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:21.402829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a216f4f-5f3b-405d-98cc-fd98e8b7601e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:21.410240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a216f4f-5f3b-405d-98cc-fd98e8b7601e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:21.411830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a216f4f-5f3b-405d-98cc-fd98e8b7601e None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-1a216f4f-5f3b-405d-98cc-fd98e8b7601e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:44:21.427023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:21.428063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:21.429042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:44:21.429042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:44:21.429042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:44:21.429042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:44:21.429042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:44:21.429042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:44:21.429042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:21.429042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:21.429042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 601/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:44:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 05:44:21.434880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db2b99-c82a-4022-81ee-2c34af5eed27 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:21.556684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db2b99-c82a-4022-81ee-2c34af5eed27 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:21.582001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:21.597522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa80c5-3496-4a3d-aaee-92676c3562e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:21.599157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa80c5-3496-4a3d-aaee-92676c3562e5 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:21.599157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa80c5-3496-4a3d-aaee-92676c3562e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:21.599157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa80c5-3496-4a3d-aaee-92676c3562e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:21.599860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa80c5-3496-4a3d-aaee-92676c3562e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:21.600186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa80c5-3496-4a3d-aaee-92676c3562e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:21.601816 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa80c5-3496-4a3d-aaee-92676c3562e5 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:21.620003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 617/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:44:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:21.626752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a216f4f-5f3b-405d-98cc-fd98e8b7601e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:21.737755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a216f4f-5f3b-405d-98cc-fd98e8b7601e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:21.770614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:21.777240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb808af6-d79d-4197-a6e2-c846729c72e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:21.778101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb808af6-d79d-4197-a6e2-c846729c72e0 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:21.778427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb808af6-d79d-4197-a6e2-c846729c72e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:21.778634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb808af6-d79d-4197-a6e2-c846729c72e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:21.779689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb808af6-d79d-4197-a6e2-c846729c72e0 None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=486de932d1db422ba2c3d84142f9a056) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:21.780017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb808af6-d79d-4197-a6e2-c846729c72e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:21.781448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb808af6-d79d-4197-a6e2-c846729c72e0 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:21.787512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 602/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:44:21 2022] GET /identity/v3/role_assignments?user.id=486de932d1db422ba2c3d84142f9a056&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:23.914837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96c138-e07c-4b24-a184-38c68ae492a3 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:23.915390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96c138-e07c-4b24-a184-38c68ae492a3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:23.915390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96c138-e07c-4b24-a184-38c68ae492a3 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:23.916754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 618/1220] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:23.922208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4e8a2-055c-41d6-8c96-4313cfe932da None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:23.922611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4e8a2-055c-41d6-8c96-4313cfe932da None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:23.922789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4e8a2-055c-41d6-8c96-4313cfe932da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:23.941873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-99d4e8a2-055c-41d6-8c96-4313cfe932da None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:23.943322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-99d4e8a2-055c-41d6-8c96-4313cfe932da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:24.012900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 603/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:44:24.039428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96c138-e07c-4b24-a184-38c68ae492a3 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:24.153166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96c138-e07c-4b24-a184-38c68ae492a3 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:24.191631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26a1eb-6ee9-44b0-891f-c4a03a775f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:24.192381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26a1eb-6ee9-44b0-891f-c4a03a775f2d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:24.192709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26a1eb-6ee9-44b0-891f-c4a03a775f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:24.193005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26a1eb-6ee9-44b0-891f-c4a03a775f2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:24.204224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26a1eb-6ee9-44b0-891f-c4a03a775f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-4a26a1eb-6ee9-44b0-891f-c4a03a775f2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:44:24.212198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:24.213368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:24.214400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 619/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:44:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:24.219959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4e8a2-055c-41d6-8c96-4313cfe932da None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:24.329694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4e8a2-055c-41d6-8c96-4313cfe932da None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:24.353764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:24.366556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13a90f-d6b0-4501-8240-b498c3f1fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:24.367302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13a90f-d6b0-4501-8240-b498c3f1fdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:24.367591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13a90f-d6b0-4501-8240-b498c3f1fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:24.367874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13a90f-d6b0-4501-8240-b498c3f1fdb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:24.368630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13a90f-d6b0-4501-8240-b498c3f1fdb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:24.368942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13a90f-d6b0-4501-8240-b498c3f1fdb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:24.370387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13a90f-d6b0-4501-8240-b498c3f1fdb6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:24.378733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 604/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:44:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:24.385273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26a1eb-6ee9-44b0-891f-c4a03a775f2d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:24.430613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:24.490008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26a1eb-6ee9-44b0-891f-c4a03a775f2d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:24.530833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869349c9-fb0d-40e7-b09c-04df66f0243b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:24.531871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869349c9-fb0d-40e7-b09c-04df66f0243b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:24.532190 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869349c9-fb0d-40e7-b09c-04df66f0243b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:24.532509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869349c9-fb0d-40e7-b09c-04df66f0243b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:24.545424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869349c9-fb0d-40e7-b09c-04df66f0243b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:24.547089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869349c9-fb0d-40e7-b09c-04df66f0243b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-869349c9-fb0d-40e7-b09c-04df66f0243b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:24.567033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:24.568080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:24.569110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:24.569110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:24.569110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:24.569110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:24.569110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 620/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:24.574311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13a90f-d6b0-4501-8240-b498c3f1fdb6 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:24.686205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13a90f-d6b0-4501-8240-b498c3f1fdb6 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:24.726283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131e075-0ace-4784-bcb8-9b5db266a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:24.726977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131e075-0ace-4784-bcb8-9b5db266a2bb None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:24.727211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131e075-0ace-4784-bcb8-9b5db266a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:24.727506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131e075-0ace-4784-bcb8-9b5db266a2bb None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:24.728229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131e075-0ace-4784-bcb8-9b5db266a2bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:24.728515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131e075-0ace-4784-bcb8-9b5db266a2bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:24.729948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131e075-0ace-4784-bcb8-9b5db266a2bb None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:24.742169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 605/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:24.748085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869349c9-fb0d-40e7-b09c-04df66f0243b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:24.848882 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:24.857528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869349c9-fb0d-40e7-b09c-04df66f0243b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:24.893209 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d724-6a6a-4ef7-a3db-6e89c97e8e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:24.894016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d724-6a6a-4ef7-a3db-6e89c97e8e85 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:24.894286 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d724-6a6a-4ef7-a3db-6e89c97e8e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:24.894559 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d724-6a6a-4ef7-a3db-6e89c97e8e85 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:24.905727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0d724-6a6a-4ef7-a3db-6e89c97e8e85 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:24.907187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0d724-6a6a-4ef7-a3db-6e89c97e8e85 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-f3f0d724-6a6a-4ef7-a3db-6e89c97e8e85 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:44:24.921911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:24.924889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:24.926074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 621/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:24.929740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131e075-0ace-4784-bcb8-9b5db266a2bb None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:24.983407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:25.040531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131e075-0ace-4784-bcb8-9b5db266a2bb None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:25.077266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688db690-cf78-4ad7-8181-7a492d7f0cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:25.078426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db690-cf78-4ad7-8181-7a492d7f0cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:25.078923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db690-cf78-4ad7-8181-7a492d7f0cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:25.079428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db690-cf78-4ad7-8181-7a492d7f0cf8 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:25.081049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688db690-cf78-4ad7-8181-7a492d7f0cf8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:25.081548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688db690-cf78-4ad7-8181-7a492d7f0cf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:25.083320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688db690-cf78-4ad7-8181-7a492d7f0cf8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:25.100874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 606/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 4 05:44:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:44:25.106719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d724-6a6a-4ef7-a3db-6e89c97e8e85 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:25.219191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d724-6a6a-4ef7-a3db-6e89c97e8e85 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:25.260904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f6e54-5628-4ec9-90c4-676e8cfc7fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:25.262132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f6e54-5628-4ec9-90c4-676e8cfc7fdd None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:25.262473 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f6e54-5628-4ec9-90c4-676e8cfc7fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:25.262833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f6e54-5628-4ec9-90c4-676e8cfc7fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:25.276264 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f6e54-5628-4ec9-90c4-676e8cfc7fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:25.278044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f6e54-5628-4ec9-90c4-676e8cfc7fdd None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-c01f6e54-5628-4ec9-90c4-676e8cfc7fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:25.298005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:25.299103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:25.300411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:25.300411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:25.300411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:25.300411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:25.300411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 622/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:25.305079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688db690-cf78-4ad7-8181-7a492d7f0cf8 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:25.409877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688db690-cf78-4ad7-8181-7a492d7f0cf8 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:25.412304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:25.444634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1ab18-79af-4fab-b9f3-e9e6736356a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:25.445393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1ab18-79af-4fab-b9f3-e9e6736356a2 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:25.445746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1ab18-79af-4fab-b9f3-e9e6736356a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:25.446091 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1ab18-79af-4fab-b9f3-e9e6736356a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:25.446892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1ab18-79af-4fab-b9f3-e9e6736356a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:25.447286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1ab18-79af-4fab-b9f3-e9e6736356a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:25.448793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1ab18-79af-4fab-b9f3-e9e6736356a2 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:25.460297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 607/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:25.465969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f6e54-5628-4ec9-90c4-676e8cfc7fdd None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:25.476540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:25.578135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f6e54-5628-4ec9-90c4-676e8cfc7fdd None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:25.617987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86b6df-24de-4d01-86f3-c8aaeadb8e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:25.618805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86b6df-24de-4d01-86f3-c8aaeadb8e59 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:25.619111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86b6df-24de-4d01-86f3-c8aaeadb8e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:25.619423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86b6df-24de-4d01-86f3-c8aaeadb8e59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:25.626568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86b6df-24de-4d01-86f3-c8aaeadb8e59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:25.628099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86b6df-24de-4d01-86f3-c8aaeadb8e59 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-3a86b6df-24de-4d01-86f3-c8aaeadb8e59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:44:25.644290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:25.645428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:25.646566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:44:25.646566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:44:25.646566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:44:25.646566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:44:25.646566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:44:25.646566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:44:25.646566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:25.646566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:25.646566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 623/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:44:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:25.652540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1ab18-79af-4fab-b9f3-e9e6736356a2 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:25.777436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1ab18-79af-4fab-b9f3-e9e6736356a2 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:25.813816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3c00d-d7bc-40f5-b2d2-74263214fd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:25.814520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3c00d-d7bc-40f5-b2d2-74263214fd21 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:25.814808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3c00d-d7bc-40f5-b2d2-74263214fd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:25.815085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3c00d-d7bc-40f5-b2d2-74263214fd21 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:25.816242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3c00d-d7bc-40f5-b2d2-74263214fd21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:25.816553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3c00d-d7bc-40f5-b2d2-74263214fd21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:25.818008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3c00d-d7bc-40f5-b2d2-74263214fd21 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:25.830803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 608/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:44:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:44:25.837168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86b6df-24de-4d01-86f3-c8aaeadb8e59 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:25.907996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:25.946526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86b6df-24de-4d01-86f3-c8aaeadb8e59 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:25.986119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1452e-6e8c-4f38-9dad-579e0196ac29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:25.987108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1452e-6e8c-4f38-9dad-579e0196ac29 None admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:25.987494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1452e-6e8c-4f38-9dad-579e0196ac29 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:25.987780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1452e-6e8c-4f38-9dad-579e0196ac29 None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/486de932d1db422ba2c3d84142f9a056/roles/04f8b72053764c7f8067d4b478d31e95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:26.002976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1452e-6e8c-4f38-9dad-579e0196ac29 None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=486de932d1db422ba2c3d84142f9a056, role_id=04f8b72053764c7f8067d4b478d31e95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:26.004482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1452e-6e8c-4f38-9dad-579e0196ac29 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:26.018383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 624/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:44:25 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/486de932d1db422ba2c3d84142f9a056/roles/04f8b72053764c7f8067d4b478d31e95 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:44:28.053259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661736b-a83d-450a-be9a-78c0ea1c6155 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:28.053815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661736b-a83d-450a-be9a-78c0ea1c6155 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:28.053815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661736b-a83d-450a-be9a-78c0ea1c6155 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:28.055183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 609/1233] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:28.060750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22424a3-5151-4eff-810e-f69f9dc693a9 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:28.061216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22424a3-5151-4eff-810e-f69f9dc693a9 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:28.061310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22424a3-5151-4eff-810e-f69f9dc693a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:28.078706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c22424a3-5151-4eff-810e-f69f9dc693a9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:28.080025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c22424a3-5151-4eff-810e-f69f9dc693a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:28.154891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 625/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:44:28.178693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661736b-a83d-450a-be9a-78c0ea1c6155 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:28.205417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:28.288919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661736b-a83d-450a-be9a-78c0ea1c6155 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:28.327267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1d8b1-a541-4678-89cc-4b5ede55328e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:28.328029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1d8b1-a541-4678-89cc-4b5ede55328e None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:28.328300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1d8b1-a541-4678-89cc-4b5ede55328e None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:28.328573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1d8b1-a541-4678-89cc-4b5ede55328e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:28.339484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1d8b1-a541-4678-89cc-4b5ede55328e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-b2e1d8b1-a541-4678-89cc-4b5ede55328e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:44:28.347919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:28.349077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:28.350276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 610/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 4 05:44:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:28.355515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22424a3-5151-4eff-810e-f69f9dc693a9 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:28.405405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:28.463993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22424a3-5151-4eff-810e-f69f9dc693a9 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:28.498781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e170955-7d19-4d1a-8708-3af8665a0e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:28.501153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e170955-7d19-4d1a-8708-3af8665a0e32 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:28.501153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e170955-7d19-4d1a-8708-3af8665a0e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:28.501153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e170955-7d19-4d1a-8708-3af8665a0e32 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:28.501153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e170955-7d19-4d1a-8708-3af8665a0e32 None admin] RBAC: Adding query filter params (name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:28.501153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e170955-7d19-4d1a-8708-3af8665a0e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:28.502506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e170955-7d19-4d1a-8708-3af8665a0e32 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:28.512671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 626/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 4 05:44:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:44:28.518487 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1d8b1-a541-4678-89cc-4b5ede55328e None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:28.606300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:28.622296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1d8b1-a541-4678-89cc-4b5ede55328e None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:28.657499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb056591-9e25-4789-b29f-2879ee0bb451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:28.658442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb056591-9e25-4789-b29f-2879ee0bb451 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:28.658738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb056591-9e25-4789-b29f-2879ee0bb451 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:28.659037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb056591-9e25-4789-b29f-2879ee0bb451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:28.670548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb056591-9e25-4789-b29f-2879ee0bb451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:28.672064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb056591-9e25-4789-b29f-2879ee0bb451 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-fb056591-9e25-4789-b29f-2879ee0bb451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:28.689861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:28.691360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:28.692401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:28.692401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:28.692401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:28.692401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:28.692401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 611/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:28.697691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e170955-7d19-4d1a-8708-3af8665a0e32 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:28.803336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e170955-7d19-4d1a-8708-3af8665a0e32 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:28.813769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:28.846022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3343e2-11bc-4181-9208-3e09554b9ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:28.846707 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3343e2-11bc-4181-9208-3e09554b9ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:28.846987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3343e2-11bc-4181-9208-3e09554b9ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:28.847281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3343e2-11bc-4181-9208-3e09554b9ab3 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:28.848006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3343e2-11bc-4181-9208-3e09554b9ab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:28.848308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3343e2-11bc-4181-9208-3e09554b9ab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:28.849700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3343e2-11bc-4181-9208-3e09554b9ab3 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:28.861932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 627/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:28.867898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb056591-9e25-4789-b29f-2879ee0bb451 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:28.977335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb056591-9e25-4789-b29f-2879ee0bb451 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:29.013242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50c680-a9d6-4293-8910-ae0aac06d63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:29.014086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50c680-a9d6-4293-8910-ae0aac06d63b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:29.014390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50c680-a9d6-4293-8910-ae0aac06d63b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:29.014659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50c680-a9d6-4293-8910-ae0aac06d63b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:29.017347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:29.022944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50c680-a9d6-4293-8910-ae0aac06d63b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:29.024665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50c680-a9d6-4293-8910-ae0aac06d63b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-ec50c680-a9d6-4293-8910-ae0aac06d63b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:44:29.039737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:29.040735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:29.041697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 612/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:29.047233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3343e2-11bc-4181-9208-3e09554b9ab3 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:29.153562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3343e2-11bc-4181-9208-3e09554b9ab3 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:29.191469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355a46-6137-49f9-ab19-41ec178a6081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:29.192268 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355a46-6137-49f9-ab19-41ec178a6081 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:29.192533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355a46-6137-49f9-ab19-41ec178a6081 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:29.192888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355a46-6137-49f9-ab19-41ec178a6081 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:29.194014 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79355a46-6137-49f9-ab19-41ec178a6081 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:29.194316 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79355a46-6137-49f9-ab19-41ec178a6081 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:29.195984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79355a46-6137-49f9-ab19-41ec178a6081 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:29.215491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 628/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 4 05:44:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:29.221236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50c680-a9d6-4293-8910-ae0aac06d63b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:29.343497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50c680-a9d6-4293-8910-ae0aac06d63b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:29.380015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59ecd4-8707-4cb8-a1cd-d0dbed5e0d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:29.380772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59ecd4-8707-4cb8-a1cd-d0dbed5e0d27 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:29.380987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59ecd4-8707-4cb8-a1cd-d0dbed5e0d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:29.381273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59ecd4-8707-4cb8-a1cd-d0dbed5e0d27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:29.451639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:29.460377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59ecd4-8707-4cb8-a1cd-d0dbed5e0d27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:29.462325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59ecd4-8707-4cb8-a1cd-d0dbed5e0d27 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:29.574139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 613/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:44:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:44:29.580995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355a46-6137-49f9-ab19-41ec178a6081 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:29.612819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:29.707560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355a46-6137-49f9-ab19-41ec178a6081 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:29.748511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90ac42-f1f4-4264-9881-28b8e68e77a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:29.749460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90ac42-f1f4-4264-9881-28b8e68e77a8 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:29.749822 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90ac42-f1f4-4264-9881-28b8e68e77a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:29.750154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90ac42-f1f4-4264-9881-28b8e68e77a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:29.762959 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90ac42-f1f4-4264-9881-28b8e68e77a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:29.764533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90ac42-f1f4-4264-9881-28b8e68e77a8 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-7b90ac42-f1f4-4264-9881-28b8e68e77a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:29.786885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:29.788025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:29.789114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:29.789114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:29.789114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:29.789114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:29.789114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 629/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:29.794485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59ecd4-8707-4cb8-a1cd-d0dbed5e0d27 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:29.906302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:29.913074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59ecd4-8707-4cb8-a1cd-d0dbed5e0d27 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:29.952463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411e5fe-783a-4a63-b1eb-a525858e4380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:29.953122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411e5fe-783a-4a63-b1eb-a525858e4380 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:29.953394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411e5fe-783a-4a63-b1eb-a525858e4380 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:29.953659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411e5fe-783a-4a63-b1eb-a525858e4380 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:29.954373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411e5fe-783a-4a63-b1eb-a525858e4380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:29.954671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411e5fe-783a-4a63-b1eb-a525858e4380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:29.956085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411e5fe-783a-4a63-b1eb-a525858e4380 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:29.968823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 614/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:29.974973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90ac42-f1f4-4264-9881-28b8e68e77a8 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:30.055415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:30.077532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90ac42-f1f4-4264-9881-28b8e68e77a8 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:30.111420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4bd59-ac1c-45f8-805c-92c992da0381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:30.112171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4bd59-ac1c-45f8-805c-92c992da0381 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:30.112927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4bd59-ac1c-45f8-805c-92c992da0381 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:30.113195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4bd59-ac1c-45f8-805c-92c992da0381 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:30.120720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4bd59-ac1c-45f8-805c-92c992da0381 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:30.122159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4bd59-ac1c-45f8-805c-92c992da0381 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-bef4bd59-ac1c-45f8-805c-92c992da0381 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:44:30.137835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:30.138759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:30.139801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:44:30.139801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:44:30.139801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:44:30.139801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:44:30.139801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:44:30.139801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:44:30.139801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:30.139801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:30.139801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 630/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:44:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:30.145410 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411e5fe-783a-4a63-b1eb-a525858e4380 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:30.268677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411e5fe-783a-4a63-b1eb-a525858e4380 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:30.313139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea79c0-4751-42c4-9850-28e7fdfeb28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:30.313909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea79c0-4751-42c4-9850-28e7fdfeb28b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:30.314226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea79c0-4751-42c4-9850-28e7fdfeb28b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:30.314464 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea79c0-4751-42c4-9850-28e7fdfeb28b None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:30.315672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea79c0-4751-42c4-9850-28e7fdfeb28b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:30.316022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea79c0-4751-42c4-9850-28e7fdfeb28b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:30.317561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea79c0-4751-42c4-9850-28e7fdfeb28b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:30.330657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 615/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:44:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:44:30.338620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4bd59-ac1c-45f8-805c-92c992da0381 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:30.457147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4bd59-ac1c-45f8-805c-92c992da0381 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:30.498323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e794c1-71d3-43b1-b155-e4217aaf24dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:30.499741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e794c1-71d3-43b1-b155-e4217aaf24dd None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:30.500069 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e794c1-71d3-43b1-b155-e4217aaf24dd None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:30.500502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e794c1-71d3-43b1-b155-e4217aaf24dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:30.501346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e794c1-71d3-43b1-b155-e4217aaf24dd None admin] RBAC: Adding query filter params (role.id=04f8b72053764c7f8067d4b478d31e95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=486de932d1db422ba2c3d84142f9a056) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:30.501663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e794c1-71d3-43b1-b155-e4217aaf24dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:30.504691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e794c1-71d3-43b1-b155-e4217aaf24dd None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:30.511105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 631/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:44:30 2022] GET /identity/v3/role_assignments?user.id=486de932d1db422ba2c3d84142f9a056&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=04f8b72053764c7f8067d4b478d31e95 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:32.794511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4ef3a-64ed-46e5-aaba-c0a590008b1d None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:32.795077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4ef3a-64ed-46e5-aaba-c0a590008b1d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:32.795077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4ef3a-64ed-46e5-aaba-c0a590008b1d None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:32.796542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 616/1247] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:32.801887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9670851-b089-4648-a7d7-788228304dd7 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:32.802420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9670851-b089-4648-a7d7-788228304dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:32.802420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9670851-b089-4648-a7d7-788228304dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:32.807973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:32.821715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-f9670851-b089-4648-a7d7-788228304dd7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:32.823062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f9670851-b089-4648-a7d7-788228304dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:32.892307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 632/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:44:32.916900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4ef3a-64ed-46e5-aaba-c0a590008b1d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:32.937037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:33.036491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4ef3a-64ed-46e5-aaba-c0a590008b1d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:33.077033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b958ce-23c7-4ec2-8121-7f80b4c57a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:33.077730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b958ce-23c7-4ec2-8121-7f80b4c57a26 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:33.078044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b958ce-23c7-4ec2-8121-7f80b4c57a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:33.078356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b958ce-23c7-4ec2-8121-7f80b4c57a26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:33.093799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b958ce-23c7-4ec2-8121-7f80b4c57a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-03b958ce-23c7-4ec2-8121-7f80b4c57a26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:44:33.103050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:33.104099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:33.104975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 617/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:44:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:33.110748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9670851-b089-4648-a7d7-788228304dd7 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:33.233815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9670851-b089-4648-a7d7-788228304dd7 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:33.270491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:33.276366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de21e0-7330-49e6-8268-37fbfd961651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:33.277139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de21e0-7330-49e6-8268-37fbfd961651 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:33.277485 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de21e0-7330-49e6-8268-37fbfd961651 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:33.278115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de21e0-7330-49e6-8268-37fbfd961651 None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:33.278644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de21e0-7330-49e6-8268-37fbfd961651 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:33.279010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de21e0-7330-49e6-8268-37fbfd961651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:33.280672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de21e0-7330-49e6-8268-37fbfd961651 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:33.290657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 633/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:44:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:33.297034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b958ce-23c7-4ec2-8121-7f80b4c57a26 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:33.380988 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:33.405220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b958ce-23c7-4ec2-8121-7f80b4c57a26 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:33.446577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ec78a-3a75-45fb-afc7-074a3414aa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:33.447411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ec78a-3a75-45fb-afc7-074a3414aa5b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:33.447694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ec78a-3a75-45fb-afc7-074a3414aa5b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:33.448017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ec78a-3a75-45fb-afc7-074a3414aa5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:33.459109 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ec78a-3a75-45fb-afc7-074a3414aa5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:33.460599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ec78a-3a75-45fb-afc7-074a3414aa5b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-af7ec78a-3a75-45fb-afc7-074a3414aa5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:33.479304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:33.480270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:33.484046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:33.484046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:33.484046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:33.484046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:33.484046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 618/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:33.486420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de21e0-7330-49e6-8268-37fbfd961651 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:33.612685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de21e0-7330-49e6-8268-37fbfd961651 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:33.675570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c855d-180d-44d5-82b2-9d82c4eb659e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:33.676304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c855d-180d-44d5-82b2-9d82c4eb659e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:33.676600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c855d-180d-44d5-82b2-9d82c4eb659e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:33.676878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c855d-180d-44d5-82b2-9d82c4eb659e None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:33.677632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c855d-180d-44d5-82b2-9d82c4eb659e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:33.677969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c855d-180d-44d5-82b2-9d82c4eb659e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:33.679888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c855d-180d-44d5-82b2-9d82c4eb659e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:33.697575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 634/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:33.706383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ec78a-3a75-45fb-afc7-074a3414aa5b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:33.825472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ec78a-3a75-45fb-afc7-074a3414aa5b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:33.854011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:33.862762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a8de-fdd3-4e9a-aea3-052fb29bccb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:33.863725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a8de-fdd3-4e9a-aea3-052fb29bccb9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:33.864086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a8de-fdd3-4e9a-aea3-052fb29bccb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:33.864365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a8de-fdd3-4e9a-aea3-052fb29bccb9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:33.872976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5a8de-fdd3-4e9a-aea3-052fb29bccb9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:33.874428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5a8de-fdd3-4e9a-aea3-052fb29bccb9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-59c5a8de-fdd3-4e9a-aea3-052fb29bccb9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:44:33.889248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:33.890424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:33.891459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 619/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:33.896606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c855d-180d-44d5-82b2-9d82c4eb659e None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:33.962222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:34.015600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c855d-180d-44d5-82b2-9d82c4eb659e None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:34.054458 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b898821-2338-48c1-8453-cee3b7f05347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:34.055233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b898821-2338-48c1-8453-cee3b7f05347 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:34.055544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b898821-2338-48c1-8453-cee3b7f05347 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:34.055931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b898821-2338-48c1-8453-cee3b7f05347 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:34.056822 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b898821-2338-48c1-8453-cee3b7f05347 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:34.057205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b898821-2338-48c1-8453-cee3b7f05347 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:34.059060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b898821-2338-48c1-8453-cee3b7f05347 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:34.081321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 635/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 4 05:44:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:34.088250 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a8de-fdd3-4e9a-aea3-052fb29bccb9 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:34.196988 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a8de-fdd3-4e9a-aea3-052fb29bccb9 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:34.232573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36675450-3146-487b-b5ca-0664cfb1e70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:34.233408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36675450-3146-487b-b5ca-0664cfb1e70b None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:34.233704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36675450-3146-487b-b5ca-0664cfb1e70b None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:34.234083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36675450-3146-487b-b5ca-0664cfb1e70b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:34.296724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:34.309501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36675450-3146-487b-b5ca-0664cfb1e70b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:34.310956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36675450-3146-487b-b5ca-0664cfb1e70b None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:34.423871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 620/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:44:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:44:34.429844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b898821-2338-48c1-8453-cee3b7f05347 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:34.540883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b898821-2338-48c1-8453-cee3b7f05347 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:34.555145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:34.577385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a87aa1-4129-467c-b5b1-e11e53423d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:34.578179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a87aa1-4129-467c-b5b1-e11e53423d28 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:34.578452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a87aa1-4129-467c-b5b1-e11e53423d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:34.578731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a87aa1-4129-467c-b5b1-e11e53423d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:34.591528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a87aa1-4129-467c-b5b1-e11e53423d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:34.593168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a87aa1-4129-467c-b5b1-e11e53423d28 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-66a87aa1-4129-467c-b5b1-e11e53423d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:34.616402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:34.617530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:34.618455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:34.618455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:34.618455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:34.618455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:34.624933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 636/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:34.626336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36675450-3146-487b-b5ca-0664cfb1e70b None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:34.726038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:34.738997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36675450-3146-487b-b5ca-0664cfb1e70b None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:34.775708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57c2b-7d68-40c0-901e-6f8c4bb54874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:34.776414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57c2b-7d68-40c0-901e-6f8c4bb54874 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:34.776711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57c2b-7d68-40c0-901e-6f8c4bb54874 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:34.777039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57c2b-7d68-40c0-901e-6f8c4bb54874 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:34.777904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c57c2b-7d68-40c0-901e-6f8c4bb54874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:34.778207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c57c2b-7d68-40c0-901e-6f8c4bb54874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:34.779816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c57c2b-7d68-40c0-901e-6f8c4bb54874 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:34.793662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 621/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:34.799711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a87aa1-4129-467c-b5b1-e11e53423d28 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:34.912797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a87aa1-4129-467c-b5b1-e11e53423d28 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:34.949848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5105f-d8db-4a77-b9a7-82c05fb7026b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:34.950687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5105f-d8db-4a77-b9a7-82c05fb7026b None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:34.950962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5105f-d8db-4a77-b9a7-82c05fb7026b None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:34.951235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5105f-d8db-4a77-b9a7-82c05fb7026b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:34.958522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5105f-d8db-4a77-b9a7-82c05fb7026b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:34.960183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5105f-d8db-4a77-b9a7-82c05fb7026b None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-ebd5105f-d8db-4a77-b9a7-82c05fb7026b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:44:34.975118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:34.976301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:34.977252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:44:34.977252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:44:34.977252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:44:34.977252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:44:34.977252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:44:34.977252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:44:34.977252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:34.977252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:34.977252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 637/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:44:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:34.982652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57c2b-7d68-40c0-901e-6f8c4bb54874 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:35.090549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57c2b-7d68-40c0-901e-6f8c4bb54874 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:35.126205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc9bc9-b1f9-4eb7-a6fa-d76a7960afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:35.126871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc9bc9-b1f9-4eb7-a6fa-d76a7960afb9 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:35.127148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc9bc9-b1f9-4eb7-a6fa-d76a7960afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:35.127459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc9bc9-b1f9-4eb7-a6fa-d76a7960afb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:35.128254 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc9bc9-b1f9-4eb7-a6fa-d76a7960afb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:35.128557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc9bc9-b1f9-4eb7-a6fa-d76a7960afb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:35.129994 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc9bc9-b1f9-4eb7-a6fa-d76a7960afb9 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:35.142760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 622/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:44:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:35.148575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5105f-d8db-4a77-b9a7-82c05fb7026b None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:35.176699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:35.266560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5105f-d8db-4a77-b9a7-82c05fb7026b None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:35.306144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e91a4-db72-429f-9d31-13e6f4cfb875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:35.306791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e91a4-db72-429f-9d31-13e6f4cfb875 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:35.307066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e91a4-db72-429f-9d31-13e6f4cfb875 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:35.307392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e91a4-db72-429f-9d31-13e6f4cfb875 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:35.308247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e91a4-db72-429f-9d31-13e6f4cfb875 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=486de932d1db422ba2c3d84142f9a056) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:35.308550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e91a4-db72-429f-9d31-13e6f4cfb875 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:35.309992 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e91a4-db72-429f-9d31-13e6f4cfb875 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:35.316343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 638/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:44:35 2022] GET /identity/v3/role_assignments?user.id=486de932d1db422ba2c3d84142f9a056&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:37.331683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202bd98-4b7c-4c04-8f22-ed908586f2ec None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:37.332245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202bd98-4b7c-4c04-8f22-ed908586f2ec None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:37.332245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202bd98-4b7c-4c04-8f22-ed908586f2ec None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:37.333632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 623/1261] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:37.340018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ce85c-df5d-483d-8f27-b2f3388e3814 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:37.340727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ce85c-df5d-483d-8f27-b2f3388e3814 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:37.340864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ce85c-df5d-483d-8f27-b2f3388e3814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:37.364333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d85ce85c-df5d-483d-8f27-b2f3388e3814 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:37.365522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d85ce85c-df5d-483d-8f27-b2f3388e3814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:37.438759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 639/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:44:37.461895 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202bd98-4b7c-4c04-8f22-ed908586f2ec None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:37.474027 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:37.573194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202bd98-4b7c-4c04-8f22-ed908586f2ec None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:37.608199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7b6c8-1f23-4eb9-be46-54ca35ce6696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:37.608914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7b6c8-1f23-4eb9-be46-54ca35ce6696 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:37.609193 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7b6c8-1f23-4eb9-be46-54ca35ce6696 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:37.609458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7b6c8-1f23-4eb9-be46-54ca35ce6696 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:37.620477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7b6c8-1f23-4eb9-be46-54ca35ce6696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-6cb7b6c8-1f23-4eb9-be46-54ca35ce6696 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:44:37.628004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:37.629051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:37.629921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 624/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:44:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:37.636194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ce85c-df5d-483d-8f27-b2f3388e3814 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:37.657223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:37.742764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ce85c-df5d-483d-8f27-b2f3388e3814 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:37.783807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1085c09-3616-42a7-bbef-5dd91d42794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:37.784814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1085c09-3616-42a7-bbef-5dd91d42794d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:37.785222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1085c09-3616-42a7-bbef-5dd91d42794d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:37.785665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1085c09-3616-42a7-bbef-5dd91d42794d None admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:37.786792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1085c09-3616-42a7-bbef-5dd91d42794d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:37.787232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1085c09-3616-42a7-bbef-5dd91d42794d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:37.789464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1085c09-3616-42a7-bbef-5dd91d42794d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:37.803115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 640/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:44:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:37.809443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7b6c8-1f23-4eb9-be46-54ca35ce6696 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:37.888541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:37.928650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7b6c8-1f23-4eb9-be46-54ca35ce6696 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:37.963363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37ba7f-8d76-4e92-9194-74ea0932abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:37.964158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37ba7f-8d76-4e92-9194-74ea0932abfe None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:37.964476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37ba7f-8d76-4e92-9194-74ea0932abfe None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:37.964749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37ba7f-8d76-4e92-9194-74ea0932abfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:37.975711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37ba7f-8d76-4e92-9194-74ea0932abfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:37.977154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37ba7f-8d76-4e92-9194-74ea0932abfe None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-cf37ba7f-8d76-4e92-9194-74ea0932abfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:37.994413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:37.995476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:37.996457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:37.996457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:37.996457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:37.996457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:37.996457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 625/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:38.001708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1085c09-3616-42a7-bbef-5dd91d42794d None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:38.087857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:38.110524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1085c09-3616-42a7-bbef-5dd91d42794d None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:38.146059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ccb4c-4b82-4ac6-b606-8904a4600e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:38.146649 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ccb4c-4b82-4ac6-b606-8904a4600e11 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:38.146930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ccb4c-4b82-4ac6-b606-8904a4600e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:38.147202 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ccb4c-4b82-4ac6-b606-8904a4600e11 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:38.147983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ccb4c-4b82-4ac6-b606-8904a4600e11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:38.148287 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ccb4c-4b82-4ac6-b606-8904a4600e11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:38.149681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ccb4c-4b82-4ac6-b606-8904a4600e11 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:38.161640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 641/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:38.167624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37ba7f-8d76-4e92-9194-74ea0932abfe None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:38.278778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37ba7f-8d76-4e92-9194-74ea0932abfe None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:38.302240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:38.318770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa6dd-3de7-4c91-a36b-248cd1e3cac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:38.319614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa6dd-3de7-4c91-a36b-248cd1e3cac7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:38.319898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa6dd-3de7-4c91-a36b-248cd1e3cac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:38.320171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa6dd-3de7-4c91-a36b-248cd1e3cac7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:38.327893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa6dd-3de7-4c91-a36b-248cd1e3cac7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:38.329347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa6dd-3de7-4c91-a36b-248cd1e3cac7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-c0ffa6dd-3de7-4c91-a36b-248cd1e3cac7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:44:38.343712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:38.345242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:38.346451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 626/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:38.351730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ccb4c-4b82-4ac6-b606-8904a4600e11 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:38.463785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ccb4c-4b82-4ac6-b606-8904a4600e11 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:38.498979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b552de-ced6-44af-b981-c55bd9d3c967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:38.499803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b552de-ced6-44af-b981-c55bd9d3c967 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:38.500067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b552de-ced6-44af-b981-c55bd9d3c967 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:38.500362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b552de-ced6-44af-b981-c55bd9d3c967 None admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:38.501203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b552de-ced6-44af-b981-c55bd9d3c967 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:38.501513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b552de-ced6-44af-b981-c55bd9d3c967 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:38.502966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b552de-ced6-44af-b981-c55bd9d3c967 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:38.512224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:38.524878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 642/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 4 05:44:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:38.530997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa6dd-3de7-4c91-a36b-248cd1e3cac7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:38.639466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa6dd-3de7-4c91-a36b-248cd1e3cac7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:38.673820 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd99b1a-ced9-4501-b9d9-5da7267a3c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:38.674611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd99b1a-ced9-4501-b9d9-5da7267a3c37 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:38.674894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd99b1a-ced9-4501-b9d9-5da7267a3c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:38.675168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd99b1a-ced9-4501-b9d9-5da7267a3c37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:38.686055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd99b1a-ced9-4501-b9d9-5da7267a3c37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:38.687533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd99b1a-ced9-4501-b9d9-5da7267a3c37 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-cdd99b1a-ced9-4501-b9d9-5da7267a3c37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:38.705335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:38.706397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:38.707397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:38.707397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:38.707397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:38.707397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:38.707397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 627/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:38.715239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b552de-ced6-44af-b981-c55bd9d3c967 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:38.821857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b552de-ced6-44af-b981-c55bd9d3c967 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:38.861357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11ce19-9b8e-4d4d-bf0c-d8599c9576ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:38.862018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11ce19-9b8e-4d4d-bf0c-d8599c9576ee None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:38.862296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11ce19-9b8e-4d4d-bf0c-d8599c9576ee None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:38.862562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11ce19-9b8e-4d4d-bf0c-d8599c9576ee None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:38.863295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11ce19-9b8e-4d4d-bf0c-d8599c9576ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:38.863609 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11ce19-9b8e-4d4d-bf0c-d8599c9576ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:38.864991 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11ce19-9b8e-4d4d-bf0c-d8599c9576ee None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:38.876484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 643/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:38.885254 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd99b1a-ced9-4501-b9d9-5da7267a3c37 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:38.918313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:38.986546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd99b1a-ced9-4501-b9d9-5da7267a3c37 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:39.024216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce5ce7-b105-4155-a747-81e607b0b0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:39.025420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce5ce7-b105-4155-a747-81e607b0b0cd None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:39.025848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce5ce7-b105-4155-a747-81e607b0b0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:39.026327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce5ce7-b105-4155-a747-81e607b0b0cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:39.033608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce5ce7-b105-4155-a747-81e607b0b0cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:39.035068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce5ce7-b105-4155-a747-81e607b0b0cd None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-92ce5ce7-b105-4155-a747-81e607b0b0cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:44:39.049659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:39.050704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:39.052028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:44:39.052028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:44:39.052028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:44:39.052028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:44:39.052028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:44:39.052028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:44:39.052028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:39.052028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:39.052028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 628/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:44:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:39.056739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11ce19-9b8e-4d4d-bf0c-d8599c9576ee None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:39.120951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:39.176678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11ce19-9b8e-4d4d-bf0c-d8599c9576ee None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:39.212262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff571e-88a6-4ba6-83db-884e49d4aa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:39.212912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff571e-88a6-4ba6-83db-884e49d4aa09 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:39.213189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff571e-88a6-4ba6-83db-884e49d4aa09 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:39.213487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff571e-88a6-4ba6-83db-884e49d4aa09 None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:39.214377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff571e-88a6-4ba6-83db-884e49d4aa09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:39.214666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff571e-88a6-4ba6-83db-884e49d4aa09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:39.216139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff571e-88a6-4ba6-83db-884e49d4aa09 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:39.229379 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 644/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:44:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:39.236422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce5ce7-b105-4155-a747-81e607b0b0cd None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:39.341870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:39.357782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce5ce7-b105-4155-a747-81e607b0b0cd None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:39.398032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66540d1e-9b3e-4412-a65e-5d705c3e569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:39.398922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540d1e-9b3e-4412-a65e-5d705c3e569c None admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:39.399203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540d1e-9b3e-4412-a65e-5d705c3e569c None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:39.399508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540d1e-9b3e-4412-a65e-5d705c3e569c None admin] PATH_INFO: `/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/486de932d1db422ba2c3d84142f9a056/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:39.401700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66540d1e-9b3e-4412-a65e-5d705c3e569c None admin] RBAC: Authorizing `identity:create_grant(project_id=b95f166a586c4a5fa22e04f9a5dc55fc, user_id=486de932d1db422ba2c3d84142f9a056, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:39.403151 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66540d1e-9b3e-4412-a65e-5d705c3e569c None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:39.423052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 629/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 4 05:44:39 2022] PUT /identity/v3/projects/b95f166a586c4a5fa22e04f9a5dc55fc/users/486de932d1db422ba2c3d84142f9a056/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:44:41.427783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324b20e-a984-427d-a8ea-ef7db26f345f None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:41.428350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324b20e-a984-427d-a8ea-ef7db26f345f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:41.428350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324b20e-a984-427d-a8ea-ef7db26f345f None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:41.429710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 645/1274] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:41.436183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c840c8-f59d-4cf5-84ee-26e1bccf522a None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:41.436913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c840c8-f59d-4cf5-84ee-26e1bccf522a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:41.437051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c840c8-f59d-4cf5-84ee-26e1bccf522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:41.464777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-13c840c8-f59d-4cf5-84ee-26e1bccf522a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:41.466028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-13c840c8-f59d-4cf5-84ee-26e1bccf522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:41.535623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 630/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:44:41.559155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324b20e-a984-427d-a8ea-ef7db26f345f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:41.680160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324b20e-a984-427d-a8ea-ef7db26f345f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:41.697769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:41.716736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e689c-1814-4c5f-a0a7-9cc9310f2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:41.717522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e689c-1814-4c5f-a0a7-9cc9310f2fbe None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:41.717800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e689c-1814-4c5f-a0a7-9cc9310f2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:41.718071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e689c-1814-4c5f-a0a7-9cc9310f2fbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:41.729120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e689c-1814-4c5f-a0a7-9cc9310f2fbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-951e689c-1814-4c5f-a0a7-9cc9310f2fbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 05:44:41.736754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:41.737820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:41.740163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 646/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 4 05:44:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:41.745725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c840c8-f59d-4cf5-84ee-26e1bccf522a None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:41.832654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:41.852031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c840c8-f59d-4cf5-84ee-26e1bccf522a None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:41.888348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92632609-849b-4e2f-92d2-952e6ba8beb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:41.889031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92632609-849b-4e2f-92d2-952e6ba8beb1 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:41.889309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92632609-849b-4e2f-92d2-952e6ba8beb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:41.889584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92632609-849b-4e2f-92d2-952e6ba8beb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:41.890333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92632609-849b-4e2f-92d2-952e6ba8beb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:41.890638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92632609-849b-4e2f-92d2-952e6ba8beb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:41.892050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92632609-849b-4e2f-92d2-952e6ba8beb1 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:41.900553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 631/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 4 05:44:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:41.906793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e689c-1814-4c5f-a0a7-9cc9310f2fbe None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:42.010237 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e689c-1814-4c5f-a0a7-9cc9310f2fbe None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:42.049417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b369e-732c-4dc3-98fe-e5adf094c741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:42.050213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b369e-732c-4dc3-98fe-e5adf094c741 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:42.050496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b369e-732c-4dc3-98fe-e5adf094c741 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:42.050764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b369e-732c-4dc3-98fe-e5adf094c741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:42.063614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b369e-732c-4dc3-98fe-e5adf094c741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:42.065459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b369e-732c-4dc3-98fe-e5adf094c741 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-c36b369e-732c-4dc3-98fe-e5adf094c741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:42.082893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:42.084022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:42.085143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:42.085143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:42.085143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:42.085143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:42.085143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 647/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:42.090738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92632609-849b-4e2f-92d2-952e6ba8beb1 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:42.212108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92632609-849b-4e2f-92d2-952e6ba8beb1 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:42.249641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b749f-bd4f-4384-bf6e-5fc5dc874728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:42.250317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b749f-bd4f-4384-bf6e-5fc5dc874728 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:42.250630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b749f-bd4f-4384-bf6e-5fc5dc874728 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:42.250879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b749f-bd4f-4384-bf6e-5fc5dc874728 None admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:42.251643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b749f-bd4f-4384-bf6e-5fc5dc874728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:42.251950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b749f-bd4f-4384-bf6e-5fc5dc874728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:42.253355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b749f-bd4f-4384-bf6e-5fc5dc874728 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:42.265440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 632/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:42.271387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b369e-732c-4dc3-98fe-e5adf094c741 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:42.304430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:42.377520 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b369e-732c-4dc3-98fe-e5adf094c741 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:42.412362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa389d6e-061b-462f-8cfd-b2ed6b132184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:42.413129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa389d6e-061b-462f-8cfd-b2ed6b132184 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:42.413401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa389d6e-061b-462f-8cfd-b2ed6b132184 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:42.413698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa389d6e-061b-462f-8cfd-b2ed6b132184 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:42.421152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa389d6e-061b-462f-8cfd-b2ed6b132184 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:42.422604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa389d6e-061b-462f-8cfd-b2ed6b132184 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-fa389d6e-061b-462f-8cfd-b2ed6b132184 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 05:44:42.436801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:42.440181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:42.441384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 648/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:42.445479 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b749f-bd4f-4384-bf6e-5fc5dc874728 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:42.449690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:42.551523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b749f-bd4f-4384-bf6e-5fc5dc874728 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:42.588382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec45194-8587-4486-a81b-d2e34fad51d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:42.589058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec45194-8587-4486-a81b-d2e34fad51d8 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:42.589341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec45194-8587-4486-a81b-d2e34fad51d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:42.589614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec45194-8587-4486-a81b-d2e34fad51d8 None admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:42.590456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec45194-8587-4486-a81b-d2e34fad51d8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:42.590763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec45194-8587-4486-a81b-d2e34fad51d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:42.592244 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec45194-8587-4486-a81b-d2e34fad51d8 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:42.610465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 633/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 4 05:44:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:42.617180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa389d6e-061b-462f-8cfd-b2ed6b132184 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:42.719339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:42.733725 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa389d6e-061b-462f-8cfd-b2ed6b132184 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:42.768645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733fc7-7e6d-4d1e-ab19-cdd310e82ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:42.769215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733fc7-7e6d-4d1e-ab19-cdd310e82ead None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:42.769494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733fc7-7e6d-4d1e-ab19-cdd310e82ead None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:42.769770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733fc7-7e6d-4d1e-ab19-cdd310e82ead None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:42.844502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09733fc7-7e6d-4d1e-ab19-cdd310e82ead None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:42.845974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09733fc7-7e6d-4d1e-ab19-cdd310e82ead None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:42.937904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:42.949788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 649/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 4 05:44:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 05:44:42.956111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec45194-8587-4486-a81b-d2e34fad51d8 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:43.019502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:43.057734 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec45194-8587-4486-a81b-d2e34fad51d8 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:43.092785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8bcd6-4f6a-41e3-b81c-686abf373d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:43.093579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8bcd6-4f6a-41e3-b81c-686abf373d15 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:43.093856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8bcd6-4f6a-41e3-b81c-686abf373d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:43.094128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8bcd6-4f6a-41e3-b81c-686abf373d15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:43.105051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8bcd6-4f6a-41e3-b81c-686abf373d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:43.106546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8bcd6-4f6a-41e3-b81c-686abf373d15 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-0ac8bcd6-4f6a-41e3-b81c-686abf373d15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 05:44:43.124639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:43.125823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:43.127112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 05:44:43.127112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 05:44:43.127112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 05:44:43.127112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:43.127112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 634/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:44:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:43.131915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733fc7-7e6d-4d1e-ab19-cdd310e82ead None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:43.237059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733fc7-7e6d-4d1e-ab19-cdd310e82ead None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:43.274488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f85b98-9ca5-4075-bba0-4160416f0a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:43.275075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f85b98-9ca5-4075-bba0-4160416f0a30 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:43.275385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f85b98-9ca5-4075-bba0-4160416f0a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:43.275695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f85b98-9ca5-4075-bba0-4160416f0a30 None admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:43.276412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f85b98-9ca5-4075-bba0-4160416f0a30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:43.276717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f85b98-9ca5-4075-bba0-4160416f0a30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:43.278097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f85b98-9ca5-4075-bba0-4160416f0a30 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:43.289868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 650/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 4 05:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:43.295940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8bcd6-4f6a-41e3-b81c-686abf373d15 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:43.399104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8bcd6-4f6a-41e3-b81c-686abf373d15 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:43.419341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:43.441890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9903d-26c4-4502-92ea-8a73739f06b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:43.442768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9903d-26c4-4502-92ea-8a73739f06b7 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:43.443039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9903d-26c4-4502-92ea-8a73739f06b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:43.443339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9903d-26c4-4502-92ea-8a73739f06b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:43.450591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9903d-26c4-4502-92ea-8a73739f06b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:43.452078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9903d-26c4-4502-92ea-8a73739f06b7 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application [None req-b7e9903d-26c4-4502-92ea-8a73739f06b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 05:44:43.466327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:43.467440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:43.468359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 05:44:43.468359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 05:44:43.468359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 05:44:43.468359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 05:44:43.468359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 05:44:43.468359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 05:44:43.468359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 05:44:43.468359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: ERROR keystone.server.flask.application Jul 04 05:44:43.468359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 635/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 4 05:44:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:43.473632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f85b98-9ca5-4075-bba0-4160416f0a30 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:43.536755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:43.590656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f85b98-9ca5-4075-bba0-4160416f0a30 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:43.625391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037391ba-228d-4d6e-adae-0bd07b6c6c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:43.626018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037391ba-228d-4d6e-adae-0bd07b6c6c7d None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:43.626294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037391ba-228d-4d6e-adae-0bd07b6c6c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:43.626564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037391ba-228d-4d6e-adae-0bd07b6c6c7d None admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:43.627560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037391ba-228d-4d6e-adae-0bd07b6c6c7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:43.627834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037391ba-228d-4d6e-adae-0bd07b6c6c7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:43.629303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037391ba-228d-4d6e-adae-0bd07b6c6c7d None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:43.641433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 651/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 4 05:44:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:43.647922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9903d-26c4-4502-92ea-8a73739f06b7 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:43.762786 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9903d-26c4-4502-92ea-8a73739f06b7 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:43.802352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0029c4-05ef-48c4-937a-6bc447c381f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:43.803008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0029c4-05ef-48c4-937a-6bc447c381f5 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:43.803294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0029c4-05ef-48c4-937a-6bc447c381f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:43.803589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0029c4-05ef-48c4-937a-6bc447c381f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:43.804423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0029c4-05ef-48c4-937a-6bc447c381f5 None admin] RBAC: Adding query filter params (role.id=8d5c4a5f0b2a44c2934090dd98122f95, scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc, user.id=486de932d1db422ba2c3d84142f9a056) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:43.804731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0029c4-05ef-48c4-937a-6bc447c381f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:43.806197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0029c4-05ef-48c4-937a-6bc447c381f5 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:43.815353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 636/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 4 05:44:43 2022] GET /identity/v3/role_assignments?user.id=486de932d1db422ba2c3d84142f9a056&scope.project.id=b95f166a586c4a5fa22e04f9a5dc55fc&role.id=8d5c4a5f0b2a44c2934090dd98122f95 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:45.835894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a6991-2989-405e-a3c4-f2266a87ab5e None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:45.836507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a6991-2989-405e-a3c4-f2266a87ab5e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:45.836507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a6991-2989-405e-a3c4-f2266a87ab5e None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:45.838432 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 652/1288] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:45.844549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7137131-89b4-48b6-93d0-3e3707c50204 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:45.845022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7137131-89b4-48b6-93d0-3e3707c50204 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:45.845154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7137131-89b4-48b6-93d0-3e3707c50204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:45.863431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d7137131-89b4-48b6-93d0-3e3707c50204 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:45.868359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d7137131-89b4-48b6-93d0-3e3707c50204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:45.897460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:45.938109 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 637/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 05:44:45.961041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a6991-2989-405e-a3c4-f2266a87ab5e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:46.067510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a6991-2989-405e-a3c4-f2266a87ab5e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:46.078216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:46.110741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f63bb8-c652-457e-841d-b50279862034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:46.111500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f63bb8-c652-457e-841d-b50279862034 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:46.111774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f63bb8-c652-457e-841d-b50279862034 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:46.112037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f63bb8-c652-457e-841d-b50279862034 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:46.112662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f63bb8-c652-457e-841d-b50279862034 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:46.114036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f63bb8-c652-457e-841d-b50279862034 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-47f63bb8-c652-457e-841d-b50279862034 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:46.126388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:46.127628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:46.128680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 653/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 4 05:44:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:46.134485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7137131-89b4-48b6-93d0-3e3707c50204 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:46.243172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7137131-89b4-48b6-93d0-3e3707c50204 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:46.284368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b8d5a-1b4e-42eb-84cc-12b8732078a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:46.285100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b8d5a-1b4e-42eb-84cc-12b8732078a3 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:46.285485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b8d5a-1b4e-42eb-84cc-12b8732078a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:46.285784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b8d5a-1b4e-42eb-84cc-12b8732078a3 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:46.286535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b8d5a-1b4e-42eb-84cc-12b8732078a3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:46.286883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b8d5a-1b4e-42eb-84cc-12b8732078a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:46.288395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b8d5a-1b4e-42eb-84cc-12b8732078a3 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:46.294630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 638/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jul 4 05:44:46 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:44:46.301741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f63bb8-c652-457e-841d-b50279862034 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:46.404545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f63bb8-c652-457e-841d-b50279862034 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:46.440045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6587c6-fdeb-4a41-8d53-5fa7132aa648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:46.440694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6587c6-fdeb-4a41-8d53-5fa7132aa648 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:46.440969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6587c6-fdeb-4a41-8d53-5fa7132aa648 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:46.441245 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6587c6-fdeb-4a41-8d53-5fa7132aa648 None admin] PATH_INFO: `/v3/services` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:46.441946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6587c6-fdeb-4a41-8d53-5fa7132aa648 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:46.442247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6587c6-fdeb-4a41-8d53-5fa7132aa648 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:46.443661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6587c6-fdeb-4a41-8d53-5fa7132aa648 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:46.448479 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 654/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jul 4 05:44:46 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:48.447619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcf9e8-a02d-4f2d-80bd-796848f09e6d None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:48.448214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcf9e8-a02d-4f2d-80bd-796848f09e6d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:48.448214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcf9e8-a02d-4f2d-80bd-796848f09e6d None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:48.449593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 639/1293] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:48.456020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52461f48-718e-4b28-a75d-05269671c43c None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:48.456739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52461f48-718e-4b28-a75d-05269671c43c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:48.456880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52461f48-718e-4b28-a75d-05269671c43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:48.482529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-52461f48-718e-4b28-a75d-05269671c43c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:48.483777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-52461f48-718e-4b28-a75d-05269671c43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:48.518234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:48.557904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 655/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:44:48.581755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcf9e8-a02d-4f2d-80bd-796848f09e6d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:48.623768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:48.704049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcf9e8-a02d-4f2d-80bd-796848f09e6d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:48.740008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669228cb-cf43-479e-8d36-5df4cf84f01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:48.740683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669228cb-cf43-479e-8d36-5df4cf84f01a None admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:48.740974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669228cb-cf43-479e-8d36-5df4cf84f01a None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:48.741240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669228cb-cf43-479e-8d36-5df4cf84f01a None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:48.741962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669228cb-cf43-479e-8d36-5df4cf84f01a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:48.743390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669228cb-cf43-479e-8d36-5df4cf84f01a None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:48.759922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 640/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 4 05:44:48 2022] POST /identity/v3/services => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:44:50.788854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac984dec-6275-44aa-b1a6-46beadd4edfa None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:50.789401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac984dec-6275-44aa-b1a6-46beadd4edfa None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:50.789401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac984dec-6275-44aa-b1a6-46beadd4edfa None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:50.790717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 656/1296] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:50.797303 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74123df-bb26-427e-8cfc-7a0e933f4c0e None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:50.798032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74123df-bb26-427e-8cfc-7a0e933f4c0e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:50.798175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74123df-bb26-427e-8cfc-7a0e933f4c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:50.825546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c74123df-bb26-427e-8cfc-7a0e933f4c0e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:50.826745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c74123df-bb26-427e-8cfc-7a0e933f4c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:50.902972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 641/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:44:50.931234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac984dec-6275-44aa-b1a6-46beadd4edfa None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:51.045506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac984dec-6275-44aa-b1a6-46beadd4edfa None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:51.084133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4a35d-ed56-4c2a-972f-7b5c08aad909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:51.084932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4a35d-ed56-4c2a-972f-7b5c08aad909 None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:51.085212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4a35d-ed56-4c2a-972f-7b5c08aad909 None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:51.085480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4a35d-ed56-4c2a-972f-7b5c08aad909 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:51.086130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4a35d-ed56-4c2a-972f-7b5c08aad909 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:51.087543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4a35d-ed56-4c2a-972f-7b5c08aad909 None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-62f4a35d-ed56-4c2a-972f-7b5c08aad909 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:51.099602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:51.100651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:51.101506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 657/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 4 05:44:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:51.107230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74123df-bb26-427e-8cfc-7a0e933f4c0e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:51.127492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:51.215867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74123df-bb26-427e-8cfc-7a0e933f4c0e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:51.258832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8e970-b510-4bba-b586-998fea965596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:51.259805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8e970-b510-4bba-b586-998fea965596 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:51.260088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8e970-b510-4bba-b586-998fea965596 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:51.260396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8e970-b510-4bba-b586-998fea965596 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:51.261492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8e970-b510-4bba-b586-998fea965596 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:51.261853 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8e970-b510-4bba-b586-998fea965596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:51.263651 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8e970-b510-4bba-b586-998fea965596 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:51.269756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 642/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jul 4 05:44:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:51.277682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4a35d-ed56-4c2a-972f-7b5c08aad909 None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:51.313442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:51.389623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4a35d-ed56-4c2a-972f-7b5c08aad909 None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:51.428849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3f7be-9069-4ba4-995e-bb8a4f63a03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:51.429521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3f7be-9069-4ba4-995e-bb8a4f63a03e None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:51.429699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3f7be-9069-4ba4-995e-bb8a4f63a03e None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:51.430766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3f7be-9069-4ba4-995e-bb8a4f63a03e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:51.430766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3f7be-9069-4ba4-995e-bb8a4f63a03e None admin] RBAC: Adding query filter params (interface=public, service_id=2d3d92d106ce4f63a880bf1d2819df1f, region_id=RegionOne) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:51.431038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3f7be-9069-4ba4-995e-bb8a4f63a03e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:51.432467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3f7be-9069-4ba4-995e-bb8a4f63a03e None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:51.437764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 658/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 4 05:44:51 2022] GET /identity/v3/endpoints?service_id=2d3d92d106ce4f63a880bf1d2819df1f&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:51.447565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8e970-b510-4bba-b586-998fea965596 None admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:51.533687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:51.554056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8e970-b510-4bba-b586-998fea965596 None admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:51.603927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff108250-a3ca-40ab-bb99-9c3e4da91bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:51.604861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff108250-a3ca-40ab-bb99-9c3e4da91bbf None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:51.605140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff108250-a3ca-40ab-bb99-9c3e4da91bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:51.605414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff108250-a3ca-40ab-bb99-9c3e4da91bbf None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:51.606184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff108250-a3ca-40ab-bb99-9c3e4da91bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:51.607616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff108250-a3ca-40ab-bb99-9c3e4da91bbf None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:51.614887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 643/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 4 05:44:51 2022] GET /identity/v3/services => generated 2191 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:44:53.608513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c140e-868c-4f56-8cf7-a9d5e10a9902 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:53.609091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c140e-868c-4f56-8cf7-a9d5e10a9902 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:53.609091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c140e-868c-4f56-8cf7-a9d5e10a9902 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:53.610429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 659/1302] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:44:53.616969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6cb55-18f3-430b-8da6-707192960b0c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:53.617737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6cb55-18f3-430b-8da6-707192960b0c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:53.617851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6cb55-18f3-430b-8da6-707192960b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:53.644153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-3ed6cb55-18f3-430b-8da6-707192960b0c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:44:53.646096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3ed6cb55-18f3-430b-8da6-707192960b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:44:53.720096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 644/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:44:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 05:44:53.753033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c140e-868c-4f56-8cf7-a9d5e10a9902 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:53.855913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:53.864232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c140e-868c-4f56-8cf7-a9d5e10a9902 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:53.911931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd73cc-1ec9-4fe2-b864-b9a5a4bffb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:53.913291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd73cc-1ec9-4fe2-b864-b9a5a4bffb7a None admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:53.913791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd73cc-1ec9-4fe2-b864-b9a5a4bffb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:53.914279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd73cc-1ec9-4fe2-b864-b9a5a4bffb7a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:53.915547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd73cc-1ec9-4fe2-b864-b9a5a4bffb7a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:53.918309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd73cc-1ec9-4fe2-b864-b9a5a4bffb7a None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application [None req-d4dd73cc-1ec9-4fe2-b864-b9a5a4bffb7a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 05:44:53.931564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 05:44:53.932704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application with Lock( Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self._enter() Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application return self.creator() Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application created_value = creator( Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: ERROR keystone.server.flask.application Jul 04 05:44:53.933694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 660/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 4 05:44:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 05:44:53.940170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6cb55-18f3-430b-8da6-707192960b0c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:54.028889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:44:54.057784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6cb55-18f3-430b-8da6-707192960b0c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:54.093828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6039e3-9b4d-4f0e-84b4-5c9f2a021dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:54.094488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6039e3-9b4d-4f0e-84b4-5c9f2a021dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:54.094767 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6039e3-9b4d-4f0e-84b4-5c9f2a021dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:54.095037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6039e3-9b4d-4f0e-84b4-5c9f2a021dd6 None admin] PATH_INFO: `/v3/services` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:54.095785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6039e3-9b4d-4f0e-84b4-5c9f2a021dd6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:44:54.096088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6039e3-9b4d-4f0e-84b4-5c9f2a021dd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:54.097465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6039e3-9b4d-4f0e-84b4-5c9f2a021dd6 None admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:54.102711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 645/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jul 4 05:44:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:44:54.108958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd73cc-1ec9-4fe2-b864-b9a5a4bffb7a None admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:44:54.236231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd73cc-1ec9-4fe2-b864-b9a5a4bffb7a None admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:44:54.274558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16208b9d-e6f5-47ac-b477-a420873e824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:44:54.275320 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16208b9d-e6f5-47ac-b477-a420873e824f None admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:44:54.275606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16208b9d-e6f5-47ac-b477-a420873e824f None admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:44:54.275970 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16208b9d-e6f5-47ac-b477-a420873e824f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:44:54.276752 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16208b9d-e6f5-47ac-b477-a420873e824f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:44:54.278333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16208b9d-e6f5-47ac-b477-a420873e824f None admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:44:54.298310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 661/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 4 05:44:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:45:04.099625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=30 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:45:04.294292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:45:57.447617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579aa17-09ef-4f65-ae8a-766a75acea50 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:45:57.448199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579aa17-09ef-4f65-ae8a-766a75acea50 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:45:57.448199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579aa17-09ef-4f65-ae8a-766a75acea50 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:45:57.453470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 646/1307] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:45:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:45:57.454860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00860f-c1d8-40d9-8c30-1c1cf5ca76e7 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:45:57.455339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00860f-c1d8-40d9-8c30-1c1cf5ca76e7 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:45:57.455426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00860f-c1d8-40d9-8c30-1c1cf5ca76e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:45:57.542509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-fb00860f-c1d8-40d9-8c30-1c1cf5ca76e7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:45:57.543713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-fb00860f-c1d8-40d9-8c30-1c1cf5ca76e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:45:57.663484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 662/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:45:57.724171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd835d-7e8e-48ae-bb99-876263ecc791 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:45:57.724818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd835d-7e8e-48ae-bb99-876263ecc791 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:45:57.725205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd835d-7e8e-48ae-bb99-876263ecc791 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:45:57.728384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 647/1309] 10.222.0.42 () {54 vars in 964 bytes} [Mon Jul 4 05:45:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:45:57.737450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883aefb-2d10-4c53-a347-65a2d7642732 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:45:57.738142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883aefb-2d10-4c53-a347-65a2d7642732 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:45:57.738870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883aefb-2d10-4c53-a347-65a2d7642732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:45:57.861055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b883aefb-2d10-4c53-a347-65a2d7642732 None None] MFA Rules not processed for user `817b1a026f4b443daea514338ad8531a`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:45:57.862571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b883aefb-2d10-4c53-a347-65a2d7642732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:45:57.885322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:45:57.943286 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 663/1310] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Jul 4 05:45:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:45:57.952434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd835d-7e8e-48ae-bb99-876263ecc791 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:45:57.954210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1bcd835d-7e8e-48ae-bb99-876263ecc791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:45:58.088787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd835d-7e8e-48ae-bb99-876263ecc791 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:45:58.174372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a624e29-4fcf-40fe-ac76-d3ced762d98e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:45:58.175040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624e29-4fcf-40fe-ac76-d3ced762d98e service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:45:58.175471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624e29-4fcf-40fe-ac76-d3ced762d98e service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:45:58.175715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624e29-4fcf-40fe-ac76-d3ced762d98e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:45:58.177406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1a624e29-4fcf-40fe-ac76-d3ced762d98e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:45:58.237195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:45:58.297815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a624e29-4fcf-40fe-ac76-d3ced762d98e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:45:58.299459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a624e29-4fcf-40fe-ac76-d3ced762d98e service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:45:58.319293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 648/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:00.656946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20775bee-8b35-4f2a-a636-cc1479b04360 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:00.657539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20775bee-8b35-4f2a-a636-cc1479b04360 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:00.657539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20775bee-8b35-4f2a-a636-cc1479b04360 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:00.659060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 664/1312] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:00.664630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc42db-31cd-4fd9-b3d9-591037e816fb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:00.665040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc42db-31cd-4fd9-b3d9-591037e816fb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:00.665189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc42db-31cd-4fd9-b3d9-591037e816fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:00.757906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-60bc42db-31cd-4fd9-b3d9-591037e816fb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:00.759333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-60bc42db-31cd-4fd9-b3d9-591037e816fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:00.867454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 649/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:00.906094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20775bee-8b35-4f2a-a636-cc1479b04360 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:00.907677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-20775bee-8b35-4f2a-a636-cc1479b04360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:01.014512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20775bee-8b35-4f2a-a636-cc1479b04360 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:01.089035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:01.092021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6828981-e7a1-446e-a93b-1ffdae8934ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:01.092780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6828981-e7a1-446e-a93b-1ffdae8934ba service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:01.093069 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6828981-e7a1-446e-a93b-1ffdae8934ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:01.093319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6828981-e7a1-446e-a93b-1ffdae8934ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:01.095436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b6828981-e7a1-446e-a93b-1ffdae8934ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:01.207230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6828981-e7a1-446e-a93b-1ffdae8934ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:01.208712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6828981-e7a1-446e-a93b-1ffdae8934ba service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:01.226337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 665/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:03.561825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7f4f-e9bb-46ae-bad4-a16366e2483d None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:03.562411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7f4f-e9bb-46ae-bad4-a16366e2483d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:03.562411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7f4f-e9bb-46ae-bad4-a16366e2483d None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:03.563787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 650/1315] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:03.569318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb04b2-f7fb-4d6c-9632-6670c3566c42 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:03.569719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb04b2-f7fb-4d6c-9632-6670c3566c42 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:03.569909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb04b2-f7fb-4d6c-9632-6670c3566c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:03.640762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-0feb04b2-f7fb-4d6c-9632-6670c3566c42 None None] MFA Rules not processed for user `b052f32177f94f9ea753f4978551e6df`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:03.642030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0feb04b2-f7fb-4d6c-9632-6670c3566c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:03.704387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:03.752295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 666/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:03 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:03.805596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659ed09-5bc8-498e-985a-f26ca3eb9aba None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:03.806075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659ed09-5bc8-498e-985a-f26ca3eb9aba None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:03.806426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659ed09-5bc8-498e-985a-f26ca3eb9aba None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:03.808059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 651/1317] 10.222.0.42 () {54 vars in 964 bytes} [Mon Jul 4 05:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:03.814511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e5bba-41b5-4d2d-9958-8843c1dcfee3 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:03.814801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e5bba-41b5-4d2d-9958-8843c1dcfee3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:03.815080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e5bba-41b5-4d2d-9958-8843c1dcfee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:03.899603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-eb4e5bba-41b5-4d2d-9958-8843c1dcfee3 None None] MFA Rules not processed for user `817b1a026f4b443daea514338ad8531a`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:03.900825 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-eb4e5bba-41b5-4d2d-9958-8843c1dcfee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:03.977048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 667/1318] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Jul 4 05:46:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:03.987011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659ed09-5bc8-498e-985a-f26ca3eb9aba None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:03.989829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b659ed09-5bc8-498e-985a-f26ca3eb9aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:04.001982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:04.102442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659ed09-5bc8-498e-985a-f26ca3eb9aba None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:04.177138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2ad5c-049a-4998-8219-3faf3b961847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:04.177796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2ad5c-049a-4998-8219-3faf3b961847 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:04.178081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2ad5c-049a-4998-8219-3faf3b961847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:04.178353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2ad5c-049a-4998-8219-3faf3b961847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:04.180040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-05d2ad5c-049a-4998-8219-3faf3b961847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:04.247677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:04.295635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2ad5c-049a-4998-8219-3faf3b961847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:04.297088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2ad5c-049a-4998-8219-3faf3b961847 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:04.316091 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 652/1319] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:03 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:07.267550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432b6c7-4186-4be1-9d60-4df238dadeb7 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:07.268156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432b6c7-4186-4be1-9d60-4df238dadeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:07.268156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432b6c7-4186-4be1-9d60-4df238dadeb7 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:07.270107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 668/1320] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:07.277899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1a616-e982-4374-9995-ab497636a1bb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:07.278397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1a616-e982-4374-9995-ab497636a1bb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:07.278493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1a616-e982-4374-9995-ab497636a1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:07.349797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-fdb1a616-e982-4374-9995-ab497636a1bb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:07.351018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fdb1a616-e982-4374-9995-ab497636a1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:07.457133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 653/1321] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:46:07.505977 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432b6c7-4186-4be1-9d60-4df238dadeb7 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:07.507550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7432b6c7-4186-4be1-9d60-4df238dadeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:07.561836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:07.617672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432b6c7-4186-4be1-9d60-4df238dadeb7 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:07.695215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc262e-d709-48f4-bafb-10f476851ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:07.695889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc262e-d709-48f4-bafb-10f476851ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:07.696164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc262e-d709-48f4-bafb-10f476851ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:07.696440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc262e-d709-48f4-bafb-10f476851ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:07.698104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c5cc262e-d709-48f4-bafb-10f476851ce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:07.806160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:07.821203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc262e-d709-48f4-bafb-10f476851ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:07.822663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc262e-d709-48f4-bafb-10f476851ce9 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:07.843030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 669/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:10.562259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ed86a-c757-47ff-877c-d202bd853e41 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:10.562876 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ed86a-c757-47ff-877c-d202bd853e41 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:10.562876 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ed86a-c757-47ff-877c-d202bd853e41 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:10.564238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 654/1323] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:10.575985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48bf36-92d3-49bd-a351-6f66de17d9c0 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:10.576502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48bf36-92d3-49bd-a351-6f66de17d9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:10.576605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48bf36-92d3-49bd-a351-6f66de17d9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:10.645180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-aa48bf36-92d3-49bd-a351-6f66de17d9c0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:10.646439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-aa48bf36-92d3-49bd-a351-6f66de17d9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:10.759007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 670/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:10.795350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ed86a-c757-47ff-877c-d202bd853e41 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:10.796942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5d3ed86a-c757-47ff-877c-d202bd853e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:10.817931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:10.907281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ed86a-c757-47ff-877c-d202bd853e41 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:10.984226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43ff6b-6794-443a-8bc9-33aa4e126519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:10.984877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43ff6b-6794-443a-8bc9-33aa4e126519 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:10.985156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43ff6b-6794-443a-8bc9-33aa4e126519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:10.985432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43ff6b-6794-443a-8bc9-33aa4e126519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:10.987102 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1b43ff6b-6794-443a-8bc9-33aa4e126519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:11.060681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:11.100477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43ff6b-6794-443a-8bc9-33aa4e126519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:11.102116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43ff6b-6794-443a-8bc9-33aa4e126519 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:11.120597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 655/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:46:13.220265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb505a-fcb6-40ee-8dc3-315b0ca6888b None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:13.220823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb505a-fcb6-40ee-8dc3-315b0ca6888b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:13.220823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb505a-fcb6-40ee-8dc3-315b0ca6888b None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:13.222055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 671/1326] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:13.227467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabbe8d-5b1f-4900-b9cf-d7a2b4125ede None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:13.227901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabbe8d-5b1f-4900-b9cf-d7a2b4125ede None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:13.227989 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabbe8d-5b1f-4900-b9cf-d7a2b4125ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:13.296305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-bfabbe8d-5b1f-4900-b9cf-d7a2b4125ede None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:13.297520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-bfabbe8d-5b1f-4900-b9cf-d7a2b4125ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:13.400476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 656/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:13.436406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb505a-fcb6-40ee-8dc3-315b0ca6888b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:13.437945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-16bb505a-fcb6-40ee-8dc3-315b0ca6888b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:13.499756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:13.549766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb505a-fcb6-40ee-8dc3-315b0ca6888b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:13.633510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d82a6d-b869-4c93-a3aa-5df32f7deeef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:13.634173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d82a6d-b869-4c93-a3aa-5df32f7deeef service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:13.634455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d82a6d-b869-4c93-a3aa-5df32f7deeef service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:13.634732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d82a6d-b869-4c93-a3aa-5df32f7deeef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:13.636429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-74d82a6d-b869-4c93-a3aa-5df32f7deeef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:13.752456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:13.754989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d82a6d-b869-4c93-a3aa-5df32f7deeef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:13.756756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d82a6d-b869-4c93-a3aa-5df32f7deeef service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:13.776857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 672/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:15.845947 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd37ae-aa05-4a3c-a126-cea5c5716a3f None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:15.846499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd37ae-aa05-4a3c-a126-cea5c5716a3f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:15.846499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd37ae-aa05-4a3c-a126-cea5c5716a3f None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:15.847871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 657/1329] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:15.853802 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a376a-7e35-4739-917e-66cad97afdb4 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:15.854296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a376a-7e35-4739-917e-66cad97afdb4 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:15.854489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a376a-7e35-4739-917e-66cad97afdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:15.925233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6d8a376a-7e35-4739-917e-66cad97afdb4 None None] MFA Rules not processed for user `b052f32177f94f9ea753f4978551e6df`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:15.926497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6d8a376a-7e35-4739-917e-66cad97afdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:16.032607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 673/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:16.069849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd37ae-aa05-4a3c-a126-cea5c5716a3f None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:16.071451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-58fd37ae-aa05-4a3c-a126-cea5c5716a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:16.103460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:16.182816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd37ae-aa05-4a3c-a126-cea5c5716a3f None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:16.257945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00c457-19e2-40bc-88fe-028ec7804bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:16.258599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00c457-19e2-40bc-88fe-028ec7804bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:16.258875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00c457-19e2-40bc-88fe-028ec7804bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:16.259141 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00c457-19e2-40bc-88fe-028ec7804bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:16.260821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6e00c457-19e2-40bc-88fe-028ec7804bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:16.350949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:16.387233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00c457-19e2-40bc-88fe-028ec7804bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:16.388736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00c457-19e2-40bc-88fe-028ec7804bb6 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:16.408943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 658/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:18.712729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856740f1-6a11-4a88-83e8-6ca5d1463467 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:18.713354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856740f1-6a11-4a88-83e8-6ca5d1463467 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:18.713354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856740f1-6a11-4a88-83e8-6ca5d1463467 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:18.714695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 674/1332] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:18.721174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d5235-72b6-4c12-86a5-ed81986e4e36 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:18.721948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d5235-72b6-4c12-86a5-ed81986e4e36 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:18.721948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d5235-72b6-4c12-86a5-ed81986e4e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:18.802263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-889d5235-72b6-4c12-86a5-ed81986e4e36 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:18.803526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-889d5235-72b6-4c12-86a5-ed81986e4e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:18.906721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 659/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:18.942850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856740f1-6a11-4a88-83e8-6ca5d1463467 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:18.944436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-856740f1-6a11-4a88-83e8-6ca5d1463467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:19.018022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:19.052975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856740f1-6a11-4a88-83e8-6ca5d1463467 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:19.131772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab742bf6-4e7f-4ee7-b00d-0c9d25f954fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:19.132437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab742bf6-4e7f-4ee7-b00d-0c9d25f954fa service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:19.132710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab742bf6-4e7f-4ee7-b00d-0c9d25f954fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:19.132981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab742bf6-4e7f-4ee7-b00d-0c9d25f954fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:19.134660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ab742bf6-4e7f-4ee7-b00d-0c9d25f954fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:19.248811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab742bf6-4e7f-4ee7-b00d-0c9d25f954fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:19.250280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab742bf6-4e7f-4ee7-b00d-0c9d25f954fa service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:19.260579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:19.270207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 675/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:21.358299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335205-03b3-4b38-a492-61545704d65a None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:21.358866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335205-03b3-4b38-a492-61545704d65a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:21.358866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335205-03b3-4b38-a492-61545704d65a None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:21.360864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 660/1335] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:21.367840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0269b-5bec-4c93-8624-d2e10cf3825d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:21.368567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0269b-5bec-4c93-8624-d2e10cf3825d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:21.368713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0269b-5bec-4c93-8624-d2e10cf3825d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:21.441570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a0e0269b-5bec-4c93-8624-d2e10cf3825d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:21.442710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a0e0269b-5bec-4c93-8624-d2e10cf3825d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:21.555506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 676/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:21.601199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae335205-03b3-4b38-a492-61545704d65a None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:21.603742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ae335205-03b3-4b38-a492-61545704d65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:21.651659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:21.717796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae335205-03b3-4b38-a492-61545704d65a None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:21.803342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c8253-16e7-4b0c-a343-62f9aeaf2107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:21.804007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8253-16e7-4b0c-a343-62f9aeaf2107 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:21.804287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8253-16e7-4b0c-a343-62f9aeaf2107 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:21.804577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8253-16e7-4b0c-a343-62f9aeaf2107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:21.806293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c69c8253-16e7-4b0c-a343-62f9aeaf2107 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:21.894539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:21.922421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c8253-16e7-4b0c-a343-62f9aeaf2107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:21.923924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c8253-16e7-4b0c-a343-62f9aeaf2107 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:21.944031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 661/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:24.430147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765f475-5187-4d0b-83e0-8a61f6a2a81c None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:24.430678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765f475-5187-4d0b-83e0-8a61f6a2a81c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:24.430678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765f475-5187-4d0b-83e0-8a61f6a2a81c None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:24.432163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 677/1338] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:24.437829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aa490-8db0-44a9-990d-f61d532f41af None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:24.438544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aa490-8db0-44a9-990d-f61d532f41af None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:24.438544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aa490-8db0-44a9-990d-f61d532f41af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:24.508277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-267aa490-8db0-44a9-990d-f61d532f41af None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:24.509512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-267aa490-8db0-44a9-990d-f61d532f41af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:24.611733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 662/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:46:24.655145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765f475-5187-4d0b-83e0-8a61f6a2a81c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:24.657442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4765f475-5187-4d0b-83e0-8a61f6a2a81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:24.757011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:24.786301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765f475-5187-4d0b-83e0-8a61f6a2a81c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:24.863333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846dd57-73ee-4701-b2ba-deaee1569359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:24.864016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846dd57-73ee-4701-b2ba-deaee1569359 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:24.864297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846dd57-73ee-4701-b2ba-deaee1569359 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:24.864571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846dd57-73ee-4701-b2ba-deaee1569359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:24.866245 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e846dd57-73ee-4701-b2ba-deaee1569359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:24.985767 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846dd57-73ee-4701-b2ba-deaee1569359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:24.987219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846dd57-73ee-4701-b2ba-deaee1569359 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:25.006729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 678/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:29.112748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63831c-b8be-4e91-9211-daeee02fb541 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:29.113416 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63831c-b8be-4e91-9211-daeee02fb541 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:29.113416 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63831c-b8be-4e91-9211-daeee02fb541 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:29.114836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 663/1341] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:29.120364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16885a3-ca43-4f7e-9777-1e5a726a34a4 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:29.120851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16885a3-ca43-4f7e-9777-1e5a726a34a4 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:29.121001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16885a3-ca43-4f7e-9777-1e5a726a34a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:29.127576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:29.198172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-e16885a3-ca43-4f7e-9777-1e5a726a34a4 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:29.199457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e16885a3-ca43-4f7e-9777-1e5a726a34a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:29.313464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 679/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:29.352238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63831c-b8be-4e91-9211-daeee02fb541 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:29.354492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-9e63831c-b8be-4e91-9211-daeee02fb541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:29.404325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:29.451973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63831c-b8be-4e91-9211-daeee02fb541 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:29.455708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02eb14-24c9-48cd-820c-b794a5891f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:29.456862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02eb14-24c9-48cd-820c-b794a5891f59 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:29.457149 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02eb14-24c9-48cd-820c-b794a5891f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:29.457426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02eb14-24c9-48cd-820c-b794a5891f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:29.466916 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02eb14-24c9-48cd-820c-b794a5891f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:29.468411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02eb14-24c9-48cd-820c-b794a5891f59 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:29.478143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 664/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:32.127817 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05fc5a-26fb-41f1-a7b0-d3d3cdc625c7 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:32.128419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05fc5a-26fb-41f1-a7b0-d3d3cdc625c7 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:32.128419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05fc5a-26fb-41f1-a7b0-d3d3cdc625c7 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:32.129744 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 680/1344] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:32.135665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81ff8-df3f-4327-8cd7-a351e5926572 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:32.136360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81ff8-df3f-4327-8cd7-a351e5926572 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:32.136596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81ff8-df3f-4327-8cd7-a351e5926572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:32.166357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-6cc81ff8-df3f-4327-8cd7-a351e5926572 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:32.167641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6cc81ff8-df3f-4327-8cd7-a351e5926572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:32.259577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 665/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:32.298437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05fc5a-26fb-41f1-a7b0-d3d3cdc625c7 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:32.310992 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05fc5a-26fb-41f1-a7b0-d3d3cdc625c7 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:32.314981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205fdf2-23b1-47d8-9760-3afb808513de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:32.315683 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205fdf2-23b1-47d8-9760-3afb808513de service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:32.315966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205fdf2-23b1-47d8-9760-3afb808513de service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:32.316265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205fdf2-23b1-47d8-9760-3afb808513de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:32.326764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205fdf2-23b1-47d8-9760-3afb808513de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:32.328271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205fdf2-23b1-47d8-9760-3afb808513de service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:32.338920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 681/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:36.492191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee34fe5-03ea-4e2f-8bff-18fc18247b9c None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:36.492728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee34fe5-03ea-4e2f-8bff-18fc18247b9c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:36.492860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee34fe5-03ea-4e2f-8bff-18fc18247b9c None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:36.494227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 666/1347] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:36.499924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362941d9-6bf2-4cbd-8211-eb2c99945d78 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:36.500407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362941d9-6bf2-4cbd-8211-eb2c99945d78 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:36.500501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362941d9-6bf2-4cbd-8211-eb2c99945d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:36.520077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-362941d9-6bf2-4cbd-8211-eb2c99945d78 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:36.521322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-362941d9-6bf2-4cbd-8211-eb2c99945d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:36.531123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 682/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:36.569040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee34fe5-03ea-4e2f-8bff-18fc18247b9c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:36.571066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2ee34fe5-03ea-4e2f-8bff-18fc18247b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:36.584356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee34fe5-03ea-4e2f-8bff-18fc18247b9c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:36.587920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd949e52-4d8f-4e6a-b163-3f78303c3434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:36.588576 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd949e52-4d8f-4e6a-b163-3f78303c3434 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:36.588844 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd949e52-4d8f-4e6a-b163-3f78303c3434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:36.589117 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd949e52-4d8f-4e6a-b163-3f78303c3434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:36.598444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd949e52-4d8f-4e6a-b163-3f78303c3434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:36.599924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd949e52-4d8f-4e6a-b163-3f78303c3434 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:36.609349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 667/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:41.685113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee2fc6-ca5e-4f50-842f-bf28fb65c7de None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:41.685618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee2fc6-ca5e-4f50-842f-bf28fb65c7de None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:41.685618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee2fc6-ca5e-4f50-842f-bf28fb65c7de None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:41.686932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 683/1350] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 05:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:41.692212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b582321-5f71-4adf-9a8b-0c059ea1e9f6 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:41.692637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b582321-5f71-4adf-9a8b-0c059ea1e9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:41.692753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b582321-5f71-4adf-9a8b-0c059ea1e9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:41.767639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-7b582321-5f71-4adf-9a8b-0c059ea1e9f6 None None] MFA Rules not processed for user `486de932d1db422ba2c3d84142f9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:41.769130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7b582321-5f71-4adf-9a8b-0c059ea1e9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:41.797610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:41.850452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 668/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 05:46:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:46:45.968623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b65e93-03fa-4fb0-a358-9ae658393b23 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:45.969131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b65e93-03fa-4fb0-a358-9ae658393b23 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:45.969131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b65e93-03fa-4fb0-a358-9ae658393b23 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:45.970611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 684/1352] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 05:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:45.975660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416bfe0-22c1-468f-87af-9790171fcd0a None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:45.976574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416bfe0-22c1-468f-87af-9790171fcd0a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:45.977064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416bfe0-22c1-468f-87af-9790171fcd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:45.996581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-9416bfe0-22c1-468f-87af-9790171fcd0a None None] MFA Rules not processed for user `486de932d1db422ba2c3d84142f9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:45.997920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-9416bfe0-22c1-468f-87af-9790171fcd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:46.006912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 669/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 05:46:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:46.520484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=72 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:47.436582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0892ea3-2ed7-4d10-8e2c-19f148a1961c None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:47.437196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0892ea3-2ed7-4d10-8e2c-19f148a1961c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:47.437196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0892ea3-2ed7-4d10-8e2c-19f148a1961c None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:47.438704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 685/1354] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 05:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:47.444260 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898f5a-97b8-4643-b461-dfbad703cb27 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:47.444694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898f5a-97b8-4643-b461-dfbad703cb27 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:47.444868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898f5a-97b8-4643-b461-dfbad703cb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:47.465227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-bd898f5a-97b8-4643-b461-dfbad703cb27 None None] MFA Rules not processed for user `486de932d1db422ba2c3d84142f9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:47.466445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-bd898f5a-97b8-4643-b461-dfbad703cb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:47.476258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 670/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 05:46:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:51.165892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfefcb-d171-46ce-be01-7b4d4d7a3887 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:51.166462 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfefcb-d171-46ce-be01-7b4d4d7a3887 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:51.166462 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfefcb-d171-46ce-be01-7b4d4d7a3887 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:51.168977 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 686/1356] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:51.174624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd309-ae5c-481e-b465-631675bd089b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:51.175097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd309-ae5c-481e-b465-631675bd089b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:51.175474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd309-ae5c-481e-b465-631675bd089b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:51.195676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-8e0fd309-ae5c-481e-b465-631675bd089b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:51.197240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8e0fd309-ae5c-481e-b465-631675bd089b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:51.206659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 671/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:51.263188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672667d6-741a-4ed0-a6f5-426a9e73824c None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:51.263900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672667d6-741a-4ed0-a6f5-426a9e73824c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:51.264480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672667d6-741a-4ed0-a6f5-426a9e73824c None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:51.266472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 687/1358] 10.222.0.42 () {54 vars in 961 bytes} [Mon Jul 4 05:46:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:51.274009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ceebb-35d0-4440-8828-f7e0ff365f20 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:51.274294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ceebb-35d0-4440-8828-f7e0ff365f20 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:51.274747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ceebb-35d0-4440-8828-f7e0ff365f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:51.325808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ed5ceebb-35d0-4440-8828-f7e0ff365f20 None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:51.326891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ed5ceebb-35d0-4440-8828-f7e0ff365f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:51.409250 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 672/1359] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Jul 4 05:46:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:51.418108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672667d6-741a-4ed0-a6f5-426a9e73824c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:51.435420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672667d6-741a-4ed0-a6f5-426a9e73824c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:51.440899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165fc6b-7dfc-4a7c-8bd0-bd837096f1c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:51.441551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165fc6b-7dfc-4a7c-8bd0-bd837096f1c5 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:51.441827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165fc6b-7dfc-4a7c-8bd0-bd837096f1c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:51.442098 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165fc6b-7dfc-4a7c-8bd0-bd837096f1c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:51.452243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165fc6b-7dfc-4a7c-8bd0-bd837096f1c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:51.453866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165fc6b-7dfc-4a7c-8bd0-bd837096f1c5 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:51.464839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 688/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:53.696080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deddad-0a6a-4a59-968d-02dae3aedee1 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:53.696630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deddad-0a6a-4a59-968d-02dae3aedee1 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:53.696630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deddad-0a6a-4a59-968d-02dae3aedee1 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:53.698192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 673/1361] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:53.703482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12a01-8b59-4b33-8a85-31650587ec2b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:53.703963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12a01-8b59-4b33-8a85-31650587ec2b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:53.704057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12a01-8b59-4b33-8a85-31650587ec2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:53.722156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-eed12a01-8b59-4b33-8a85-31650587ec2b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:53.723415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-eed12a01-8b59-4b33-8a85-31650587ec2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:53.731843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 689/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:53.776258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deddad-0a6a-4a59-968d-02dae3aedee1 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:53.791815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deddad-0a6a-4a59-968d-02dae3aedee1 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:53.797840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee77102-db26-48db-bb4e-efaa8a772ffe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:53.798565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee77102-db26-48db-bb4e-efaa8a772ffe service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:53.798811 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee77102-db26-48db-bb4e-efaa8a772ffe service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:53.799038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee77102-db26-48db-bb4e-efaa8a772ffe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:53.807904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee77102-db26-48db-bb4e-efaa8a772ffe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:53.809866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee77102-db26-48db-bb4e-efaa8a772ffe service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:53.820118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 674/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:56.210977 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fb0ae-199b-44ed-8fbd-3a3e30ac5360 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:56.212054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fb0ae-199b-44ed-8fbd-3a3e30ac5360 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:56.212340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fb0ae-199b-44ed-8fbd-3a3e30ac5360 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:56.213974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 690/1364] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:56.220250 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719410a-9f9c-489b-b70b-fe9e2fb42fba None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:56.220723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719410a-9f9c-489b-b70b-fe9e2fb42fba None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:56.220836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719410a-9f9c-489b-b70b-fe9e2fb42fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:56.240129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-5719410a-9f9c-489b-b70b-fe9e2fb42fba None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:56.241387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5719410a-9f9c-489b-b70b-fe9e2fb42fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:56.250427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 675/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:56.295478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858ca6a-87ba-4a59-9222-1c670a8fd718 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:56.295783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858ca6a-87ba-4a59-9222-1c670a8fd718 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:56.296175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858ca6a-87ba-4a59-9222-1c670a8fd718 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:56.297586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 691/1366] 10.222.0.42 () {54 vars in 961 bytes} [Mon Jul 4 05:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:56.302945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa256f-e235-48ae-b18c-bb64a247b5a2 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:56.303236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa256f-e235-48ae-b18c-bb64a247b5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:56.303548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa256f-e235-48ae-b18c-bb64a247b5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:56.323523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d1fa256f-e235-48ae-b18c-bb64a247b5a2 None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:56.324737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d1fa256f-e235-48ae-b18c-bb64a247b5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:56.335021 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 676/1367] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Jul 4 05:46:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:56.343279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858ca6a-87ba-4a59-9222-1c670a8fd718 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:56.358241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858ca6a-87ba-4a59-9222-1c670a8fd718 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:56.364015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bfdb7-9541-4083-b6e8-c2b3049e8c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:56.364659 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bfdb7-9541-4083-b6e8-c2b3049e8c98 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:56.364936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bfdb7-9541-4083-b6e8-c2b3049e8c98 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:56.365208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bfdb7-9541-4083-b6e8-c2b3049e8c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:56.373531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bfdb7-9541-4083-b6e8-c2b3049e8c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:56.375123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bfdb7-9541-4083-b6e8-c2b3049e8c98 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:56.384732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 692/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:46:58.474670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813851f-a034-46f4-b79e-cd4b7fd089e3 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:58.475308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813851f-a034-46f4-b79e-cd4b7fd089e3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:58.475308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813851f-a034-46f4-b79e-cd4b7fd089e3 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:58.476604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 677/1369] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:58.482028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdc5d3-87b7-43ce-a732-8f49c0bb7501 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:58.482980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdc5d3-87b7-43ce-a732-8f49c0bb7501 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:58.483298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdc5d3-87b7-43ce-a732-8f49c0bb7501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:58.504028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c0bdc5d3-87b7-43ce-a732-8f49c0bb7501 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:58.505307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c0bdc5d3-87b7-43ce-a732-8f49c0bb7501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:58.517152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 693/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:58.561430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813851f-a034-46f4-b79e-cd4b7fd089e3 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:58.576234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813851f-a034-46f4-b79e-cd4b7fd089e3 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:58.581904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603049d-5dab-4db4-81c3-3fa07c23bc02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:58.582657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603049d-5dab-4db4-81c3-3fa07c23bc02 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:58.583028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603049d-5dab-4db4-81c3-3fa07c23bc02 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:58.583448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603049d-5dab-4db4-81c3-3fa07c23bc02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:58.592176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603049d-5dab-4db4-81c3-3fa07c23bc02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:58.593985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603049d-5dab-4db4-81c3-3fa07c23bc02 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:58.604059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 678/1371] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:46:59.218394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995eb61-a51c-4296-a648-75b5f5d9215e None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:59.219052 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995eb61-a51c-4296-a648-75b5f5d9215e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:59.219765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995eb61-a51c-4296-a648-75b5f5d9215e None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:59.225701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 694/1372] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:59.227772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8bed-dd95-40de-a522-0a55f71181b4 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:59.228266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8bed-dd95-40de-a522-0a55f71181b4 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:59.228722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8bed-dd95-40de-a522-0a55f71181b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:59.250358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-b22b8bed-dd95-40de-a522-0a55f71181b4 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:59.251849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b22b8bed-dd95-40de-a522-0a55f71181b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:59.259986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 679/1373] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:59.599606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1f44a-52b1-49f1-9944-c285b2de5df9 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:59.600448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1f44a-52b1-49f1-9944-c285b2de5df9 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:59.600448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1f44a-52b1-49f1-9944-c285b2de5df9 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:59.603286 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 695/1374] 10.222.0.42 () {54 vars in 963 bytes} [Mon Jul 4 05:46:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:46:59.608726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ad8d0-0b54-49f3-a5cf-e11e3b5189a9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:59.609178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ad8d0-0b54-49f3-a5cf-e11e3b5189a9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:59.609271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ad8d0-0b54-49f3-a5cf-e11e3b5189a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:59.616535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:46:59.663472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-e45ad8d0-0b54-49f3-a5cf-e11e3b5189a9 None None] MFA Rules not processed for user `0f63e110260e4471b0a54576d3573583`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:46:59.665218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e45ad8d0-0b54-49f3-a5cf-e11e3b5189a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:46:59.731072 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 680/1375] 10.222.0.42 () {58 vars in 1060 bytes} [Mon Jul 4 05:46:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:46:59.740397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1f44a-52b1-49f1-9944-c285b2de5df9 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:46:59.752315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1f44a-52b1-49f1-9944-c285b2de5df9 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:46:59.756167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43a482-8290-471f-98b6-50c3a49c49e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:46:59.756969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43a482-8290-471f-98b6-50c3a49c49e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:46:59.757346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43a482-8290-471f-98b6-50c3a49c49e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:46:59.757720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43a482-8290-471f-98b6-50c3a49c49e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:46:59.767554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43a482-8290-471f-98b6-50c3a49c49e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:46:59.769071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43a482-8290-471f-98b6-50c3a49c49e9 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:46:59.779855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 696/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:00.660657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025942a-e0ec-43d9-a2f9-67d6bff4d0a6 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:00.661214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025942a-e0ec-43d9-a2f9-67d6bff4d0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:00.661214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025942a-e0ec-43d9-a2f9-67d6bff4d0a6 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:00.662564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 681/1377] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:00.667968 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d40bc-8384-47a4-b576-4b330ace4a6d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:00.668497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d40bc-8384-47a4-b576-4b330ace4a6d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:00.668497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d40bc-8384-47a4-b576-4b330ace4a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:00.686493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6b2d40bc-8384-47a4-b576-4b330ace4a6d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:00.687745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6b2d40bc-8384-47a4-b576-4b330ace4a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:00.696108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 697/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:00.739894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025942a-e0ec-43d9-a2f9-67d6bff4d0a6 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:00.755655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025942a-e0ec-43d9-a2f9-67d6bff4d0a6 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:00.761790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6afae3-b49a-42ee-8102-7a27fe265b10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:00.762432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6afae3-b49a-42ee-8102-7a27fe265b10 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:00.762708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6afae3-b49a-42ee-8102-7a27fe265b10 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:00.762978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6afae3-b49a-42ee-8102-7a27fe265b10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:00.771653 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6afae3-b49a-42ee-8102-7a27fe265b10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:00.773240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6afae3-b49a-42ee-8102-7a27fe265b10 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:00.782814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 682/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:02.832424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e894f2-43c8-42e4-8eec-e17f0b80b5e0 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:02.832929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e894f2-43c8-42e4-8eec-e17f0b80b5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:02.832929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e894f2-43c8-42e4-8eec-e17f0b80b5e0 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:02.834243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 698/1380] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:02.839918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba197500-6aff-4c97-9ff4-0d9bf59c64c9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:02.840342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba197500-6aff-4c97-9ff4-0d9bf59c64c9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:02.840574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba197500-6aff-4c97-9ff4-0d9bf59c64c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:02.859914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ba197500-6aff-4c97-9ff4-0d9bf59c64c9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:02.861268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ba197500-6aff-4c97-9ff4-0d9bf59c64c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:02.870532 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 683/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:02.914340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e894f2-43c8-42e4-8eec-e17f0b80b5e0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:02.934811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e894f2-43c8-42e4-8eec-e17f0b80b5e0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:02.940696 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5847e-a753-4ceb-9af4-0ca239604cbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:02.941410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5847e-a753-4ceb-9af4-0ca239604cbe service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:02.941699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5847e-a753-4ceb-9af4-0ca239604cbe service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:02.941978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5847e-a753-4ceb-9af4-0ca239604cbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:02.953608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5847e-a753-4ceb-9af4-0ca239604cbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:02.956109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5847e-a753-4ceb-9af4-0ca239604cbe service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:02.967059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 699/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:05.040632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae2d91-8b8b-4de1-a0cb-dd0519f94509 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:05.041178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae2d91-8b8b-4de1-a0cb-dd0519f94509 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:05.041178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae2d91-8b8b-4de1-a0cb-dd0519f94509 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:05.042778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 684/1383] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:05.048136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e272-8192-4733-826e-366861ec400d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:05.048591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e272-8192-4733-826e-366861ec400d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:05.048704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e272-8192-4733-826e-366861ec400d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:05.074976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b157e272-8192-4733-826e-366861ec400d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:05.076202 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b157e272-8192-4733-826e-366861ec400d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:05.086526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 700/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:05.128312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae2d91-8b8b-4de1-a0cb-dd0519f94509 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:05.143547 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae2d91-8b8b-4de1-a0cb-dd0519f94509 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:05.149329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329beb59-971d-49b3-b07b-0b80135b74fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:05.149927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329beb59-971d-49b3-b07b-0b80135b74fb service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:05.150196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329beb59-971d-49b3-b07b-0b80135b74fb service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:05.150647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329beb59-971d-49b3-b07b-0b80135b74fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:05.161623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329beb59-971d-49b3-b07b-0b80135b74fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:05.163227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329beb59-971d-49b3-b07b-0b80135b74fb service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:05.173343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 685/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:07.224759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b2b9-df8e-457c-a529-db0b192763f6 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:07.225359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b2b9-df8e-457c-a529-db0b192763f6 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:07.225359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b2b9-df8e-457c-a529-db0b192763f6 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:07.226595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 701/1386] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:07.232590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b466c2-a6d2-471a-81cd-c96ba44a4492 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:07.233071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b466c2-a6d2-471a-81cd-c96ba44a4492 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:07.233163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b466c2-a6d2-471a-81cd-c96ba44a4492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:07.252535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-38b466c2-a6d2-471a-81cd-c96ba44a4492 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:07.253758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-38b466c2-a6d2-471a-81cd-c96ba44a4492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:07.262115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 686/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:07.307815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98b2b9-df8e-457c-a529-db0b192763f6 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:07.323141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98b2b9-df8e-457c-a529-db0b192763f6 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:07.329128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d794d-809a-4651-869c-5a48918b02a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:07.329750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d794d-809a-4651-869c-5a48918b02a3 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:07.330023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d794d-809a-4651-869c-5a48918b02a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:07.330297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d794d-809a-4651-869c-5a48918b02a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:07.339861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d794d-809a-4651-869c-5a48918b02a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:07.341764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d794d-809a-4651-869c-5a48918b02a3 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:07.357129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 702/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:09.650496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5ed8b-9353-4a06-956a-836369bfd01b None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:09.651183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5ed8b-9353-4a06-956a-836369bfd01b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:09.651183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5ed8b-9353-4a06-956a-836369bfd01b None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:09.652542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 687/1389] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:09.658369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ebd1a-649f-479f-9001-4c11f3b97d73 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:09.658877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ebd1a-649f-479f-9001-4c11f3b97d73 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:09.659330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ebd1a-649f-479f-9001-4c11f3b97d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:09.679857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a87ebd1a-649f-479f-9001-4c11f3b97d73 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:09.681144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a87ebd1a-649f-479f-9001-4c11f3b97d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:09.690262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 703/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:09.736993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5ed8b-9353-4a06-956a-836369bfd01b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:09.755715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5ed8b-9353-4a06-956a-836369bfd01b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:09.762005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c22298-3d67-4f6d-b217-1343f2b04dab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:09.762683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c22298-3d67-4f6d-b217-1343f2b04dab service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:09.762988 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c22298-3d67-4f6d-b217-1343f2b04dab service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:09.763326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c22298-3d67-4f6d-b217-1343f2b04dab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:09.772517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c22298-3d67-4f6d-b217-1343f2b04dab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:09.773971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c22298-3d67-4f6d-b217-1343f2b04dab service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:09.783902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 688/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:12.404312 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5a228-43df-4cfc-802a-a6b4c41a92af None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:12.404927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5a228-43df-4cfc-802a-a6b4c41a92af None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:12.404927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5a228-43df-4cfc-802a-a6b4c41a92af None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:12.406364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 704/1392] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:12.411769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71055d-3418-4521-af85-42186e6bdaf3 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:12.412209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71055d-3418-4521-af85-42186e6bdaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:12.412306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71055d-3418-4521-af85-42186e6bdaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:12.431272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ba71055d-3418-4521-af85-42186e6bdaf3 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:12.432627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ba71055d-3418-4521-af85-42186e6bdaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:12.442207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 689/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:12.496017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5a228-43df-4cfc-802a-a6b4c41a92af None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:12.511155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5a228-43df-4cfc-802a-a6b4c41a92af None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:12.516870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e192-df02-4d33-9ed1-cdfec5d4035d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:12.517926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e192-df02-4d33-9ed1-cdfec5d4035d service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:12.517926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e192-df02-4d33-9ed1-cdfec5d4035d service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:12.518142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e192-df02-4d33-9ed1-cdfec5d4035d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:12.527783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705e192-df02-4d33-9ed1-cdfec5d4035d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:12.529462 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705e192-df02-4d33-9ed1-cdfec5d4035d service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:12.539716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 705/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:14.643231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5fa0d-633a-41e2-a49b-9e8d2a7305e8 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:14.643887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5fa0d-633a-41e2-a49b-9e8d2a7305e8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:14.643887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5fa0d-633a-41e2-a49b-9e8d2a7305e8 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:14.645195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 690/1395] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:14.650586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa36c8-46cd-40cd-9610-e10ccf1982dc None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:14.651122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa36c8-46cd-40cd-9610-e10ccf1982dc None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:14.651227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa36c8-46cd-40cd-9610-e10ccf1982dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:14.669696 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-f4aa36c8-46cd-40cd-9610-e10ccf1982dc None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:14.670874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f4aa36c8-46cd-40cd-9610-e10ccf1982dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:14.679072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 706/1396] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:14.723017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5fa0d-633a-41e2-a49b-9e8d2a7305e8 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:14.738908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5fa0d-633a-41e2-a49b-9e8d2a7305e8 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:14.745077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0dcd74-194d-4118-b016-f368b54cdcea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:14.745725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0dcd74-194d-4118-b016-f368b54cdcea service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:14.746011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0dcd74-194d-4118-b016-f368b54cdcea service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:14.746287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0dcd74-194d-4118-b016-f368b54cdcea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:14.754571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0dcd74-194d-4118-b016-f368b54cdcea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:14.756213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0dcd74-194d-4118-b016-f368b54cdcea service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:14.766134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 691/1397] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:14.800645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978acfe1-4c08-4b60-afb7-a3fa511ecf0e None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:14.800930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978acfe1-4c08-4b60-afb7-a3fa511ecf0e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:14.801208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978acfe1-4c08-4b60-afb7-a3fa511ecf0e None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:14.802575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 707/1398] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:14.808323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbce8c-39c9-4edf-a0fe-f22daae38c21 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:14.808966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbce8c-39c9-4edf-a0fe-f22daae38c21 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:14.809241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbce8c-39c9-4edf-a0fe-f22daae38c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:14.828132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d1dbce8c-39c9-4edf-a0fe-f22daae38c21 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:14.829419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d1dbce8c-39c9-4edf-a0fe-f22daae38c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:14.838580 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 692/1399] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:47:14.896415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d5263-6fa8-48d0-b4f0-ab344b1bd549 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:14.896799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d5263-6fa8-48d0-b4f0-ab344b1bd549 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:14.897115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d5263-6fa8-48d0-b4f0-ab344b1bd549 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:14.898638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 708/1400] 10.222.0.42 () {54 vars in 962 bytes} [Mon Jul 4 05:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:14.904654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c0c11-5f5b-4b27-925b-00d63a1a3ca3 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:14.905075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c0c11-5f5b-4b27-925b-00d63a1a3ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:14.905305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c0c11-5f5b-4b27-925b-00d63a1a3ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:14.962714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-060c0c11-5f5b-4b27-925b-00d63a1a3ca3 None None] MFA Rules not processed for user `ab33473d40094ec681e538cde250ec2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:14.964027 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-060c0c11-5f5b-4b27-925b-00d63a1a3ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:14.997209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:47:15.024592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 693/1401] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Jul 4 05:47:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:15.032336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d5263-6fa8-48d0-b4f0-ab344b1bd549 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:15.043366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d5263-6fa8-48d0-b4f0-ab344b1bd549 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:15.047182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359c871-5fd4-4fce-85a0-5182bd7fec31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab33473d40094ec681e538cde250ec2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:15.047848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359c871-5fd4-4fce-85a0-5182bd7fec31 service glance] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:15.048125 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359c871-5fd4-4fce-85a0-5182bd7fec31 service glance] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:15.048390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359c871-5fd4-4fce-85a0-5182bd7fec31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:15.058743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359c871-5fd4-4fce-85a0-5182bd7fec31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:15.060280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359c871-5fd4-4fce-85a0-5182bd7fec31 service glance] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:15.070154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 709/1402] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:15.813289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dc0f4-f1ce-40f0-a350-713f678527d3 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:15.813772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dc0f4-f1ce-40f0-a350-713f678527d3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:15.813883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dc0f4-f1ce-40f0-a350-713f678527d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:15.865290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-369dc0f4-f1ce-40f0-a350-713f678527d3 None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:15.866499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-369dc0f4-f1ce-40f0-a350-713f678527d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:15.935675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 694/1403] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 05:47:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:15.975948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfd991-9c63-48df-95f5-9397e32329e9 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:15.976461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfd991-9c63-48df-95f5-9397e32329e9 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:15.976576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfd991-9c63-48df-95f5-9397e32329e9 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:15.978104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 710/1404] 10.222.0.42 () {54 vars in 942 bytes} [Mon Jul 4 05:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:15.984586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2c03-df3c-4d0c-9124-b16d28ccbadb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:15.984882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2c03-df3c-4d0c-9124-b16d28ccbadb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:15.985127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2c03-df3c-4d0c-9124-b16d28ccbadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:16.034392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c6db2c03-df3c-4d0c-9124-b16d28ccbadb None None] MFA Rules not processed for user `655b3c5b98fa4d91b328d04d42fc263a`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:16.035562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c6db2c03-df3c-4d0c-9124-b16d28ccbadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:16.108431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 695/1405] 10.222.0.42 () {58 vars in 1039 bytes} [Mon Jul 4 05:47:15 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:16.118018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfd991-9c63-48df-95f5-9397e32329e9 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:16.132279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfd991-9c63-48df-95f5-9397e32329e9 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:16.137792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71b719-0b45-4c55-b6bd-fb01f64bd7cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:16.138434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71b719-0b45-4c55-b6bd-fb01f64bd7cd service swift] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:16.138701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71b719-0b45-4c55-b6bd-fb01f64bd7cd service swift] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:16.138979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71b719-0b45-4c55-b6bd-fb01f64bd7cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:16.147000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71b719-0b45-4c55-b6bd-fb01f64bd7cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:16.148628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71b719-0b45-4c55-b6bd-fb01f64bd7cd service swift] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:16.156887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 711/1406] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 05:47:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 05:47:16.852212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4603c-d03a-4f6c-aa34-c4a0a59204ab None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:16.853233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4603c-d03a-4f6c-aa34-c4a0a59204ab None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:16.853695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4603c-d03a-4f6c-aa34-c4a0a59204ab None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:16.855328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 696/1407] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:16.861236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3c732-83bb-42d6-8944-9009d69b8340 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:16.861528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3c732-83bb-42d6-8944-9009d69b8340 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:16.861804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3c732-83bb-42d6-8944-9009d69b8340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:16.867808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=67 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:47:16.881697 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-02b3c732-83bb-42d6-8944-9009d69b8340 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:16.882892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-02b3c732-83bb-42d6-8944-9009d69b8340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:16.895648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 712/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:16.939107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4603c-d03a-4f6c-aa34-c4a0a59204ab None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:16.954458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4603c-d03a-4f6c-aa34-c4a0a59204ab None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:16.960360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b84df-6657-4377-acfe-89ac87c72e20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:16.961095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b84df-6657-4377-acfe-89ac87c72e20 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:16.961383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b84df-6657-4377-acfe-89ac87c72e20 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:16.961656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b84df-6657-4377-acfe-89ac87c72e20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:16.972392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b84df-6657-4377-acfe-89ac87c72e20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:16.974807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b84df-6657-4377-acfe-89ac87c72e20 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:16.981194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:47:16.989086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 697/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:26.876352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:47:38.309587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495340d5-b50d-41d1-993f-e154cd6956b8 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:38.310319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495340d5-b50d-41d1-993f-e154cd6956b8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:38.310319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495340d5-b50d-41d1-993f-e154cd6956b8 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:38.311579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 713/1410] 10.222.0.42 () {54 vars in 918 bytes} [Mon Jul 4 05:47:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:38.316819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba236f0-6dbf-4fed-9c4a-67debf0dbdf2 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:38.317260 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba236f0-6dbf-4fed-9c4a-67debf0dbdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:38.317364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba236f0-6dbf-4fed-9c4a-67debf0dbdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:38.343584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-fba236f0-6dbf-4fed-9c4a-67debf0dbdf2 None None] MFA Rules not processed for user `486de932d1db422ba2c3d84142f9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:38.345363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fba236f0-6dbf-4fed-9c4a-67debf0dbdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:38.358335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 698/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Jul 4 05:47:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:38.373482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c7aec-e263-4cea-9578-d00218379068 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:38.373771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c7aec-e263-4cea-9578-d00218379068 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:38.374041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c7aec-e263-4cea-9578-d00218379068 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:38.375454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 714/1412] 10.222.0.42 () {54 vars in 918 bytes} [Mon Jul 4 05:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:38.380142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043abe9-b0de-49b6-87a7-daa820402cbf None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:38.380416 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043abe9-b0de-49b6-87a7-daa820402cbf None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:38.380683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043abe9-b0de-49b6-87a7-daa820402cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:38.399993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-f043abe9-b0de-49b6-87a7-daa820402cbf None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:38.401197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f043abe9-b0de-49b6-87a7-daa820402cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:38.412273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 699/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Jul 4 05:47:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:47:40.916153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e521b0-de95-4867-af76-5081a0c4ecc4 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:40.916973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e521b0-de95-4867-af76-5081a0c4ecc4 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:40.916973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e521b0-de95-4867-af76-5081a0c4ecc4 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:40.919078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:47:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:40.925509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfc738-e6a7-4cf2-9932-abea5ec1e35a None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:40.925948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfc738-e6a7-4cf2-9932-abea5ec1e35a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:40.926107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfc738-e6a7-4cf2-9932-abea5ec1e35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:40.947360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-70cfc738-e6a7-4cf2-9932-abea5ec1e35a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:40.948577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-70cfc738-e6a7-4cf2-9932-abea5ec1e35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:40.957165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:41.003952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e521b0-de95-4867-af76-5081a0c4ecc4 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:41.019847 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e521b0-de95-4867-af76-5081a0c4ecc4 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:41.026232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c48f58-47c5-431f-98de-92b3cccad236 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:41.026869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c48f58-47c5-431f-98de-92b3cccad236 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:41.027158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c48f58-47c5-431f-98de-92b3cccad236 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:41.027465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c48f58-47c5-431f-98de-92b3cccad236 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:41.036608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c48f58-47c5-431f-98de-92b3cccad236 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:41.038247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c48f58-47c5-431f-98de-92b3cccad236 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:41.048282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 716/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:41.950725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f77ce-5ac3-4c71-b71e-62324a632a8d None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:41.951682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f77ce-5ac3-4c71-b71e-62324a632a8d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:41.951682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f77ce-5ac3-4c71-b71e-62324a632a8d None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:41.953802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:47:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:41.960073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5e53f-d8da-47c9-90e3-7e717544f500 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:41.960502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5e53f-d8da-47c9-90e3-7e717544f500 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:41.960617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5e53f-d8da-47c9-90e3-7e717544f500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:41.986845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a6d5e53f-d8da-47c9-90e3-7e717544f500 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:41.988081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a6d5e53f-d8da-47c9-90e3-7e717544f500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:41.997146 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:42.149440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f77ce-5ac3-4c71-b71e-62324a632a8d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:42.165583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f77ce-5ac3-4c71-b71e-62324a632a8d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:42.171681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113ad04-534b-40fc-9193-356f03aae64a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:42.172297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ad04-534b-40fc-9193-356f03aae64a service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:42.172615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ad04-534b-40fc-9193-356f03aae64a service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:42.172884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ad04-534b-40fc-9193-356f03aae64a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:42.182216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113ad04-534b-40fc-9193-356f03aae64a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:42.184198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113ad04-534b-40fc-9193-356f03aae64a service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:42.197312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 702/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:43.074492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbf21-bc28-4830-955a-778c22869a86 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:43.075100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbf21-bc28-4830-955a-778c22869a86 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:43.075100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbf21-bc28-4830-955a-778c22869a86 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:43.077273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:43.085733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f8421-a8bb-4f31-b271-ca265cc1ee08 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:43.086458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f8421-a8bb-4f31-b271-ca265cc1ee08 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:43.086611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f8421-a8bb-4f31-b271-ca265cc1ee08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:43.106363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-f60f8421-a8bb-4f31-b271-ca265cc1ee08 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:43.107621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f60f8421-a8bb-4f31-b271-ca265cc1ee08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:43.115960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:43.271151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbf21-bc28-4830-955a-778c22869a86 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:43.288906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbf21-bc28-4830-955a-778c22869a86 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:43.295164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d49572-526b-440a-9d2d-0d627595760d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:43.295838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d49572-526b-440a-9d2d-0d627595760d service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:43.296114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d49572-526b-440a-9d2d-0d627595760d service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:43.296387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d49572-526b-440a-9d2d-0d627595760d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:43.304498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d49572-526b-440a-9d2d-0d627595760d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:43.306102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d49572-526b-440a-9d2d-0d627595760d service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:43.315829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 719/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:44.179287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f051c7-37a7-488b-87dd-eb92ba4e40c3 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:44.179861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f051c7-37a7-488b-87dd-eb92ba4e40c3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:44.179861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f051c7-37a7-488b-87dd-eb92ba4e40c3 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:44.181804 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:47:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:44.188160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aedf7-14b6-41df-a52f-e9199d859026 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:44.188854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aedf7-14b6-41df-a52f-e9199d859026 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:44.189003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aedf7-14b6-41df-a52f-e9199d859026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:44.211372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-e42aedf7-14b6-41df-a52f-e9199d859026 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:44.212560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e42aedf7-14b6-41df-a52f-e9199d859026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:44.220980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:44.374120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f051c7-37a7-488b-87dd-eb92ba4e40c3 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:44.394674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f051c7-37a7-488b-87dd-eb92ba4e40c3 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:44.401186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb77c71-0419-40b3-b945-0996a9032e0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:44.401834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb77c71-0419-40b3-b945-0996a9032e0d service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:44.402115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb77c71-0419-40b3-b945-0996a9032e0d service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:44.402388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb77c71-0419-40b3-b945-0996a9032e0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:44.410910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb77c71-0419-40b3-b945-0996a9032e0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:44.412560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb77c71-0419-40b3-b945-0996a9032e0d service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:44.422321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 705/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:45.297492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04b269-020f-49a1-95b7-6b7836339b73 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:45.298111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04b269-020f-49a1-95b7-6b7836339b73 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:45.298111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04b269-020f-49a1-95b7-6b7836339b73 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:45.300098 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:47:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:45.305020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3052de1-614d-4777-b69b-09a5c85db8f9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:45.305483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3052de1-614d-4777-b69b-09a5c85db8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:45.305607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3052de1-614d-4777-b69b-09a5c85db8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:45.324393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-b3052de1-614d-4777-b69b-09a5c85db8f9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:45.325598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b3052de1-614d-4777-b69b-09a5c85db8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:45.333953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:45.491971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04b269-020f-49a1-95b7-6b7836339b73 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:45.508099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04b269-020f-49a1-95b7-6b7836339b73 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:45.514201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecb903-ee6b-4829-81e9-28852eb16ccb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:45.514856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecb903-ee6b-4829-81e9-28852eb16ccb service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:45.515137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecb903-ee6b-4829-81e9-28852eb16ccb service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:45.515449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecb903-ee6b-4829-81e9-28852eb16ccb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:45.524937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecb903-ee6b-4829-81e9-28852eb16ccb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:45.526639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecb903-ee6b-4829-81e9-28852eb16ccb service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:45.536261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 722/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:46.343682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c903eb-958b-4cce-abc5-1063f55d2b10 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:46.344296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c903eb-958b-4cce-abc5-1063f55d2b10 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:46.344296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c903eb-958b-4cce-abc5-1063f55d2b10 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:46.345641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:46.351936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f0cf3-ff48-4438-aeaa-4f71f732b384 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:46.352619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f0cf3-ff48-4438-aeaa-4f71f732b384 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:46.352851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f0cf3-ff48-4438-aeaa-4f71f732b384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:46.378777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-2c3f0cf3-ff48-4438-aeaa-4f71f732b384 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:46.380015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2c3f0cf3-ff48-4438-aeaa-4f71f732b384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:46.389436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:46.539661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c903eb-958b-4cce-abc5-1063f55d2b10 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:46.557117 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c903eb-958b-4cce-abc5-1063f55d2b10 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:46.564145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1c6f9-11a2-4b0b-9a45-87a49df6cbeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:46.564793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1c6f9-11a2-4b0b-9a45-87a49df6cbeb service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:46.565071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1c6f9-11a2-4b0b-9a45-87a49df6cbeb service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:46.565340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1c6f9-11a2-4b0b-9a45-87a49df6cbeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:46.574409 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1c6f9-11a2-4b0b-9a45-87a49df6cbeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:46.576082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1c6f9-11a2-4b0b-9a45-87a49df6cbeb service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:46.585541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 708/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:47.397360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a093b-975e-487d-a278-28554967e90e None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:47.397996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a093b-975e-487d-a278-28554967e90e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:47.397996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a093b-975e-487d-a278-28554967e90e None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:47.399341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:47.404568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caecb59-c889-4a85-b7ff-f0ad9118f0f8 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:47.405019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caecb59-c889-4a85-b7ff-f0ad9118f0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:47.405150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caecb59-c889-4a85-b7ff-f0ad9118f0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:47.424303 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-1caecb59-c889-4a85-b7ff-f0ad9118f0f8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:47.425515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1caecb59-c889-4a85-b7ff-f0ad9118f0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:47.434654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:47.589565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a093b-975e-487d-a278-28554967e90e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:47.614720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a093b-975e-487d-a278-28554967e90e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:47.621903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d7fe0-4454-4d53-9807-c1151e7228e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:47.622545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d7fe0-4454-4d53-9807-c1151e7228e5 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:47.622819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d7fe0-4454-4d53-9807-c1151e7228e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:47.623084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d7fe0-4454-4d53-9807-c1151e7228e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:47.631693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d7fe0-4454-4d53-9807-c1151e7228e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:47.633280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d7fe0-4454-4d53-9807-c1151e7228e5 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:47.643021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 725/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:48.465166 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c9638-81cb-4cc5-8d84-354d3b2672e7 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:48.465772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c9638-81cb-4cc5-8d84-354d3b2672e7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:48.465772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c9638-81cb-4cc5-8d84-354d3b2672e7 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:48.467894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:47:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:48.476332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0dfb2-0b41-4030-b226-1e587a2c3710 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:48.476733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0dfb2-0b41-4030-b226-1e587a2c3710 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:48.476949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0dfb2-0b41-4030-b226-1e587a2c3710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:48.503187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-bca0dfb2-0b41-4030-b226-1e587a2c3710 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:48.504451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-bca0dfb2-0b41-4030-b226-1e587a2c3710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:48.515233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:48.670774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c9638-81cb-4cc5-8d84-354d3b2672e7 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:48.687980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c9638-81cb-4cc5-8d84-354d3b2672e7 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:48.694771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bef00b-2320-417d-b10e-02b475e20697 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:48.695455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bef00b-2320-417d-b10e-02b475e20697 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:48.695742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bef00b-2320-417d-b10e-02b475e20697 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:48.696009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bef00b-2320-417d-b10e-02b475e20697 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:48.704765 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bef00b-2320-417d-b10e-02b475e20697 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:48.706365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bef00b-2320-417d-b10e-02b475e20697 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:48.716170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 711/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:49.527709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42936675-cf56-4f6e-8ee4-ee8bb325cc12 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:49.528196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42936675-cf56-4f6e-8ee4-ee8bb325cc12 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:49.528305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42936675-cf56-4f6e-8ee4-ee8bb325cc12 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:49.529615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:49.534677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19255889-fdfd-4b0d-914e-89c51aeebb99 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:49.535108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19255889-fdfd-4b0d-914e-89c51aeebb99 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:49.535216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19255889-fdfd-4b0d-914e-89c51aeebb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:49.553791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-19255889-fdfd-4b0d-914e-89c51aeebb99 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:49.554981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-19255889-fdfd-4b0d-914e-89c51aeebb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:49.563767 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:49.719478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42936675-cf56-4f6e-8ee4-ee8bb325cc12 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:49.735746 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42936675-cf56-4f6e-8ee4-ee8bb325cc12 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:49.742141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62edf-3988-4d8c-8274-5cb61f9ee2f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:49.742775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62edf-3988-4d8c-8274-5cb61f9ee2f9 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:49.743052 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62edf-3988-4d8c-8274-5cb61f9ee2f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:49.743398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62edf-3988-4d8c-8274-5cb61f9ee2f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:49.751808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e62edf-3988-4d8c-8274-5cb61f9ee2f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:49.753393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e62edf-3988-4d8c-8274-5cb61f9ee2f9 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:49.762939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 728/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:51.671815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3b068-d9c6-46a3-bdce-a69cbfe19939 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:51.672408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3b068-d9c6-46a3-bdce-a69cbfe19939 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:51.672408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3b068-d9c6-46a3-bdce-a69cbfe19939 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:51.673875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:51.680181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859f247-caed-4905-9f1d-3569b2b86cae None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:51.680910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859f247-caed-4905-9f1d-3569b2b86cae None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:51.681042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859f247-caed-4905-9f1d-3569b2b86cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:51.707167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-0859f247-caed-4905-9f1d-3569b2b86cae None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:51.708409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0859f247-caed-4905-9f1d-3569b2b86cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:51.717987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:51.755997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3b068-d9c6-46a3-bdce-a69cbfe19939 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:51.768064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3b068-d9c6-46a3-bdce-a69cbfe19939 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:51.772246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a1186-6dac-4dff-829e-ae03eba118ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:51.772925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a1186-6dac-4dff-829e-ae03eba118ff service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:51.773208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a1186-6dac-4dff-829e-ae03eba118ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:51.773484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a1186-6dac-4dff-829e-ae03eba118ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:51.783015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a1186-6dac-4dff-829e-ae03eba118ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:51.784508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a1186-6dac-4dff-829e-ae03eba118ff service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:51.793917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 714/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:54.009128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f5c40-9efb-4102-8d45-5f62f37b9060 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:54.009686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f5c40-9efb-4102-8d45-5f62f37b9060 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:54.009686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f5c40-9efb-4102-8d45-5f62f37b9060 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:54.011241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:54.019919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33afe04-3fe5-442a-bfdc-9e3ea8866382 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:54.020597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33afe04-3fe5-442a-bfdc-9e3ea8866382 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:54.020703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33afe04-3fe5-442a-bfdc-9e3ea8866382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:54.041335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-e33afe04-3fe5-442a-bfdc-9e3ea8866382 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:54.042632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e33afe04-3fe5-442a-bfdc-9e3ea8866382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:54.051919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:54.091036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f5c40-9efb-4102-8d45-5f62f37b9060 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:54.108941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f5c40-9efb-4102-8d45-5f62f37b9060 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:54.113334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c8228-8518-4ff9-8c02-ea04e06f7e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:54.113989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c8228-8518-4ff9-8c02-ea04e06f7e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:54.114269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c8228-8518-4ff9-8c02-ea04e06f7e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:54.114546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c8228-8518-4ff9-8c02-ea04e06f7e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:54.124476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c8228-8518-4ff9-8c02-ea04e06f7e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:54.125953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c8228-8518-4ff9-8c02-ea04e06f7e2b service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:54.135661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 731/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:56.488138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758e4e4-f9e3-4142-91d8-0bdf5e0c6aa6 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:56.488717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758e4e4-f9e3-4142-91d8-0bdf5e0c6aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:56.488717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758e4e4-f9e3-4142-91d8-0bdf5e0c6aa6 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:56.490076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:56.496455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b289e2-0d37-4987-9fdf-b5cd68d063aa None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:56.497221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b289e2-0d37-4987-9fdf-b5cd68d063aa None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:56.497413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b289e2-0d37-4987-9fdf-b5cd68d063aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:56.521690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a1b289e2-0d37-4987-9fdf-b5cd68d063aa None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:56.522911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a1b289e2-0d37-4987-9fdf-b5cd68d063aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:56.532451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:56.583845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758e4e4-f9e3-4142-91d8-0bdf5e0c6aa6 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:56.598335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758e4e4-f9e3-4142-91d8-0bdf5e0c6aa6 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:56.603113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5be15-e52a-4244-ad8f-0e5399208b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:56.603799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5be15-e52a-4244-ad8f-0e5399208b51 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:56.604085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5be15-e52a-4244-ad8f-0e5399208b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:56.604390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5be15-e52a-4244-ad8f-0e5399208b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:56.614219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5be15-e52a-4244-ad8f-0e5399208b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:56.615715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5be15-e52a-4244-ad8f-0e5399208b51 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:56.625280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 717/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:47:58.794481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdc66f-ac9e-4d9b-91a8-09e20a358d50 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:58.795385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdc66f-ac9e-4d9b-91a8-09e20a358d50 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:58.795385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdc66f-ac9e-4d9b-91a8-09e20a358d50 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:58.797604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:47:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:47:58.803219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75f92a-5e3d-409a-a9cc-76ee6c44277d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:58.803694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75f92a-5e3d-409a-a9cc-76ee6c44277d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:58.803800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75f92a-5e3d-409a-a9cc-76ee6c44277d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:58.822457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2c75f92a-5e3d-409a-a9cc-76ee6c44277d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:47:58.823733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2c75f92a-5e3d-409a-a9cc-76ee6c44277d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:47:58.832078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:47:58.872405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdc66f-ac9e-4d9b-91a8-09e20a358d50 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:47:58.884989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdc66f-ac9e-4d9b-91a8-09e20a358d50 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:47:58.889203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebefccc-cb06-4fff-b266-ad0040e9bc30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:47:58.889849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefccc-cb06-4fff-b266-ad0040e9bc30 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:47:58.890130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefccc-cb06-4fff-b266-ad0040e9bc30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:47:58.890407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefccc-cb06-4fff-b266-ad0040e9bc30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:47:58.900141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebefccc-cb06-4fff-b266-ad0040e9bc30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:47:58.901640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebefccc-cb06-4fff-b266-ad0040e9bc30 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:47:58.911371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 734/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:47:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:01.019805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007643c8-fe44-4540-a5d4-5f62e455a6ed None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:01.020368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007643c8-fe44-4540-a5d4-5f62e455a6ed None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:01.020368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007643c8-fe44-4540-a5d4-5f62e455a6ed None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:01.021761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:01.027689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc025fa-ecf0-469e-b909-ff90329287b9 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:01.028154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc025fa-ecf0-469e-b909-ff90329287b9 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:01.028251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc025fa-ecf0-469e-b909-ff90329287b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:01.048792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-8bc025fa-ecf0-469e-b909-ff90329287b9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:01.050032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8bc025fa-ecf0-469e-b909-ff90329287b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:01.059530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:01.100945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007643c8-fe44-4540-a5d4-5f62e455a6ed None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:01.119628 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007643c8-fe44-4540-a5d4-5f62e455a6ed None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:01.123735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d7c72-06f0-4e77-b63e-4e00971fc7b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:01.124501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d7c72-06f0-4e77-b63e-4e00971fc7b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:01.124787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d7c72-06f0-4e77-b63e-4e00971fc7b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:01.125048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d7c72-06f0-4e77-b63e-4e00971fc7b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:01.134237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d7c72-06f0-4e77-b63e-4e00971fc7b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:01.135727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d7c72-06f0-4e77-b63e-4e00971fc7b9 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:01.145117 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 720/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:03.237627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e3daa-1df6-46a4-9678-0e5de137b9d5 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:03.238143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e3daa-1df6-46a4-9678-0e5de137b9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:03.238143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e3daa-1df6-46a4-9678-0e5de137b9d5 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:03.239575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:48:03.245154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fd72b-a720-44d1-927b-d62034d53581 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:03.245957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fd72b-a720-44d1-927b-d62034d53581 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:03.245957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fd72b-a720-44d1-927b-d62034d53581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:03.265396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c42fd72b-a720-44d1-927b-d62034d53581 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:03.266594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c42fd72b-a720-44d1-927b-d62034d53581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:03.275213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:03.323272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e3daa-1df6-46a4-9678-0e5de137b9d5 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:03.341398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e3daa-1df6-46a4-9678-0e5de137b9d5 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:03.347772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcb1b6-e62c-4807-aeb4-f2ac0e5dfb47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:03.348657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcb1b6-e62c-4807-aeb4-f2ac0e5dfb47 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:03.349049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcb1b6-e62c-4807-aeb4-f2ac0e5dfb47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:03.349446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcb1b6-e62c-4807-aeb4-f2ac0e5dfb47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:03.365027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcb1b6-e62c-4807-aeb4-f2ac0e5dfb47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:03.367176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcb1b6-e62c-4807-aeb4-f2ac0e5dfb47 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:03.378552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 737/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:05.454367 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f461fbc-3883-4ec7-8e72-717018d121b4 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:05.454910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f461fbc-3883-4ec7-8e72-717018d121b4 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:05.454910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f461fbc-3883-4ec7-8e72-717018d121b4 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:05.456249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:05.462498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c676ae-94c5-4701-adc3-09b130d9a448 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:05.462934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c676ae-94c5-4701-adc3-09b130d9a448 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:05.463047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c676ae-94c5-4701-adc3-09b130d9a448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:05.487481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-81c676ae-94c5-4701-adc3-09b130d9a448 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:05.488698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-81c676ae-94c5-4701-adc3-09b130d9a448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:05.498000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:05.536690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f461fbc-3883-4ec7-8e72-717018d121b4 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:05.550044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f461fbc-3883-4ec7-8e72-717018d121b4 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:05.554512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b9f00-6d6f-4d9d-bfc1-9a900f8c0830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:05.555165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9f00-6d6f-4d9d-bfc1-9a900f8c0830 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:05.555507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9f00-6d6f-4d9d-bfc1-9a900f8c0830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:05.555756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9f00-6d6f-4d9d-bfc1-9a900f8c0830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:05.566214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b9f00-6d6f-4d9d-bfc1-9a900f8c0830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:05.567738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b9f00-6d6f-4d9d-bfc1-9a900f8c0830 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:05.577599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 723/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:07.653192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a584ae7-4b91-4460-8db8-c0c919702900 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:07.654177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a584ae7-4b91-4460-8db8-c0c919702900 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:07.654177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a584ae7-4b91-4460-8db8-c0c919702900 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:07.656358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:48:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:07.662721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cab4c-7693-4473-8cd0-74cb6a3a92a7 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:07.663155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cab4c-7693-4473-8cd0-74cb6a3a92a7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:07.663373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cab4c-7693-4473-8cd0-74cb6a3a92a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:07.671413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=79 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:48:07.691354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-688cab4c-7693-4473-8cd0-74cb6a3a92a7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:07.692564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-688cab4c-7693-4473-8cd0-74cb6a3a92a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:07.701475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:07.738812 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a584ae7-4b91-4460-8db8-c0c919702900 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:07.745271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=73 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:48:07.755112 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a584ae7-4b91-4460-8db8-c0c919702900 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:07.761531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3878ff2-98e1-46a6-8dd8-15f2e5255efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:07.762180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3878ff2-98e1-46a6-8dd8-15f2e5255efc service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:07.762461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3878ff2-98e1-46a6-8dd8-15f2e5255efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:07.762736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3878ff2-98e1-46a6-8dd8-15f2e5255efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:07.772269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3878ff2-98e1-46a6-8dd8-15f2e5255efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:07.773665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3878ff2-98e1-46a6-8dd8-15f2e5255efc service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:07.783873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 740/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:09.896897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11033-d261-459f-be1d-57426c6565da None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:09.897805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11033-d261-459f-be1d-57426c6565da None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:09.897805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11033-d261-459f-be1d-57426c6565da None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:09.899922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:48:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:09.908803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460e389-ae4e-45c9-bf04-29c411943f64 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:09.909252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460e389-ae4e-45c9-bf04-29c411943f64 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:09.909358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460e389-ae4e-45c9-bf04-29c411943f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:09.931003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a460e389-ae4e-45c9-bf04-29c411943f64 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:09.932240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a460e389-ae4e-45c9-bf04-29c411943f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:09.941443 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:09.977517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d11033-d261-459f-be1d-57426c6565da None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:09.990524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d11033-d261-459f-be1d-57426c6565da None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:09.994880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa8b05-6c61-4b41-a0db-e58734e29405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:09.995597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa8b05-6c61-4b41-a0db-e58734e29405 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:09.995879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa8b05-6c61-4b41-a0db-e58734e29405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:09.996154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa8b05-6c61-4b41-a0db-e58734e29405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:10.006702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa8b05-6c61-4b41-a0db-e58734e29405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:10.008202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa8b05-6c61-4b41-a0db-e58734e29405 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:10.017867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 726/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:12.045204 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c0799-2803-4139-8d35-f80c1f99c2cc None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:12.046162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c0799-2803-4139-8d35-f80c1f99c2cc None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:12.046162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c0799-2803-4139-8d35-f80c1f99c2cc None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:12.048270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:48:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:12.054657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f906f-ec13-4a59-8109-2538441f2aaa None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:12.055456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f906f-ec13-4a59-8109-2538441f2aaa None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:12.055640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f906f-ec13-4a59-8109-2538441f2aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:12.081523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-cc5f906f-ec13-4a59-8109-2538441f2aaa None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:12.082730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cc5f906f-ec13-4a59-8109-2538441f2aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:12.091831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:12.127994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c0799-2803-4139-8d35-f80c1f99c2cc None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:12.139937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c0799-2803-4139-8d35-f80c1f99c2cc None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:12.143749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534ae89-e2a3-447c-aea7-0c591310e3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:12.144393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534ae89-e2a3-447c-aea7-0c591310e3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:12.144675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534ae89-e2a3-447c-aea7-0c591310e3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:12.144951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534ae89-e2a3-447c-aea7-0c591310e3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:12.155164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534ae89-e2a3-447c-aea7-0c591310e3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:12.156649 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534ae89-e2a3-447c-aea7-0c591310e3e3 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:12.166961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 743/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:22.076007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=11 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:48:22.162028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=14 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:48:31.983478 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6df52e-33b0-4443-bf4b-a35aac30e37b None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:31.984376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6df52e-33b0-4443-bf4b-a35aac30e37b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:31.984483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6df52e-33b0-4443-bf4b-a35aac30e37b None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:31.986678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 728/1471] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:48:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:31.993073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc287e-0939-49ef-b246-ce00f7a0cc46 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:31.993962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc287e-0939-49ef-b246-ce00f7a0cc46 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:31.993962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc287e-0939-49ef-b246-ce00f7a0cc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:32.013685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-07dc287e-0939-49ef-b246-ce00f7a0cc46 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:32.014885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-07dc287e-0939-49ef-b246-ce00f7a0cc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:32.023556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 744/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:32.112659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf58db5-f779-4be5-92f9-d429f6a41acb None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:32.113008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf58db5-f779-4be5-92f9-d429f6a41acb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:32.113363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf58db5-f779-4be5-92f9-d429f6a41acb None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:32.115407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 729/1473] 10.222.0.42 () {54 vars in 962 bytes} [Mon Jul 4 05:48:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:32.121990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6876751-a62d-4ac1-a752-c62fe7137d30 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:32.122278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6876751-a62d-4ac1-a752-c62fe7137d30 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:32.122554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6876751-a62d-4ac1-a752-c62fe7137d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:32.141801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c6876751-a62d-4ac1-a752-c62fe7137d30 None None] MFA Rules not processed for user `ab33473d40094ec681e538cde250ec2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:32.143016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c6876751-a62d-4ac1-a752-c62fe7137d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:32.150778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 745/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Jul 4 05:48:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:32.158653 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf58db5-f779-4be5-92f9-d429f6a41acb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:32.173627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf58db5-f779-4be5-92f9-d429f6a41acb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:32.177748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279560c4-fb52-4d83-bf04-4b63f8e10228 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab33473d40094ec681e538cde250ec2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:32.178390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279560c4-fb52-4d83-bf04-4b63f8e10228 service glance] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:32.178672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279560c4-fb52-4d83-bf04-4b63f8e10228 service glance] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:32.178945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279560c4-fb52-4d83-bf04-4b63f8e10228 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:32.188881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279560c4-fb52-4d83-bf04-4b63f8e10228 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:32.190330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279560c4-fb52-4d83-bf04-4b63f8e10228 service glance] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:32.200059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 730/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:34.321424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb1681-7b22-403c-b077-90b8e1f2f85a None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:34.322403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb1681-7b22-403c-b077-90b8e1f2f85a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:34.322403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb1681-7b22-403c-b077-90b8e1f2f85a None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:34.324642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 746/1476] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:48:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:34.333082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235fc99-a095-4206-8867-134597b8c236 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:34.333835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235fc99-a095-4206-8867-134597b8c236 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:34.333984 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235fc99-a095-4206-8867-134597b8c236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:34.354621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2235fc99-a095-4206-8867-134597b8c236 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:34.355936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2235fc99-a095-4206-8867-134597b8c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:34.364421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 731/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:34.389173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb1681-7b22-403c-b077-90b8e1f2f85a None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:34.410351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb1681-7b22-403c-b077-90b8e1f2f85a None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:34.417852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb3e27-3b16-4ce2-8eef-fed2bfdfb5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:34.418509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb3e27-3b16-4ce2-8eef-fed2bfdfb5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:34.418793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb3e27-3b16-4ce2-8eef-fed2bfdfb5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:34.419065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb3e27-3b16-4ce2-8eef-fed2bfdfb5f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:34.419843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb3e27-3b16-4ce2-8eef-fed2bfdfb5f4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:34.421663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: WARNING py.warnings [None req-22bb3e27-3b16-4ce2-8eef-fed2bfdfb5f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:48:34.421663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: warnings.warn(msg) Jul 04 05:48:34.421663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: Jul 04 05:48:34.422829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb3e27-3b16-4ce2-8eef-fed2bfdfb5f4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:34.443307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 747/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 4 05:48:34 2022] GET /identity/v3/projects => generated 2553 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:48:36.592341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130effc-ea23-446c-bd56-580eff04312c None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:36.592909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130effc-ea23-446c-bd56-580eff04312c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:36.592909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130effc-ea23-446c-bd56-580eff04312c None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:36.597091 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 732/1479] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:48:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:36.606866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bff19-44a3-456e-a054-60cc1285cdab None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:36.607549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bff19-44a3-456e-a054-60cc1285cdab None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:36.607819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bff19-44a3-456e-a054-60cc1285cdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:36.633737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a70bff19-44a3-456e-a054-60cc1285cdab None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:36.634973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a70bff19-44a3-456e-a054-60cc1285cdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:36.648920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 748/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:36.695583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130effc-ea23-446c-bd56-580eff04312c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:36.710411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130effc-ea23-446c-bd56-580eff04312c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:36.716106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69299d5-8174-4c03-b76e-b9820437fe3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:36.716753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69299d5-8174-4c03-b76e-b9820437fe3e service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:36.717113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69299d5-8174-4c03-b76e-b9820437fe3e service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:36.717392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69299d5-8174-4c03-b76e-b9820437fe3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:36.725506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69299d5-8174-4c03-b76e-b9820437fe3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:36.727097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69299d5-8174-4c03-b76e-b9820437fe3e service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:36.736891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 733/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:38.937080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93ded0-6fe0-4190-a313-fd961de81618 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:38.937636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93ded0-6fe0-4190-a313-fd961de81618 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:38.937636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93ded0-6fe0-4190-a313-fd961de81618 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:38.939012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 749/1482] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:38.944772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df140f9-5fdf-4a3b-b088-3309a0925320 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:38.945186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df140f9-5fdf-4a3b-b088-3309a0925320 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:38.945305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df140f9-5fdf-4a3b-b088-3309a0925320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:38.965042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-7df140f9-5fdf-4a3b-b088-3309a0925320 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:38.966258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7df140f9-5fdf-4a3b-b088-3309a0925320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:38.975600 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 734/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:39.014131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93ded0-6fe0-4190-a313-fd961de81618 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:39.026864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93ded0-6fe0-4190-a313-fd961de81618 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:39.031389 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f219fa6-112e-415c-ac8b-7d0730d02588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:39.032029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f219fa6-112e-415c-ac8b-7d0730d02588 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:39.032313 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f219fa6-112e-415c-ac8b-7d0730d02588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:39.032585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f219fa6-112e-415c-ac8b-7d0730d02588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:39.045910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f219fa6-112e-415c-ac8b-7d0730d02588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:39.047387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f219fa6-112e-415c-ac8b-7d0730d02588 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:39.056898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 750/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:41.119852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240d7c5-5183-40e3-b82c-c77f8ce66e38 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:41.120474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240d7c5-5183-40e3-b82c-c77f8ce66e38 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:41.120474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240d7c5-5183-40e3-b82c-c77f8ce66e38 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:41.121870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 735/1485] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:48:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:41.128382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3983b93-b3d6-420c-bf3b-93097b2886cb None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:41.128782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3983b93-b3d6-420c-bf3b-93097b2886cb None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:41.128961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3983b93-b3d6-420c-bf3b-93097b2886cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:41.150502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b3983b93-b3d6-420c-bf3b-93097b2886cb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:41.151778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b3983b93-b3d6-420c-bf3b-93097b2886cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:41.160426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 751/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:41.200894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240d7c5-5183-40e3-b82c-c77f8ce66e38 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:41.214276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240d7c5-5183-40e3-b82c-c77f8ce66e38 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:41.218721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ec0d3-3e21-42d1-b662-128d004559d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:41.219419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ec0d3-3e21-42d1-b662-128d004559d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:41.219703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ec0d3-3e21-42d1-b662-128d004559d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:41.219979 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ec0d3-3e21-42d1-b662-128d004559d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:41.229909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ec0d3-3e21-42d1-b662-128d004559d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:41.231370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ec0d3-3e21-42d1-b662-128d004559d5 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:41.240816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 736/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:43.422246 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f01856-d26c-4582-afa7-d66287d33a96 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:43.423133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f01856-d26c-4582-afa7-d66287d33a96 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:43.423133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f01856-d26c-4582-afa7-d66287d33a96 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:43.425440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 752/1488] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:48:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:43.433761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0788bc-98ae-475f-ad1c-c35dfb51729d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:43.434205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0788bc-98ae-475f-ad1c-c35dfb51729d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:43.434315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0788bc-98ae-475f-ad1c-c35dfb51729d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:43.453726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4d0788bc-98ae-475f-ad1c-c35dfb51729d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:43.454959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4d0788bc-98ae-475f-ad1c-c35dfb51729d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:43.463569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 737/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:48:43.505912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f01856-d26c-4582-afa7-d66287d33a96 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:43.518750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f01856-d26c-4582-afa7-d66287d33a96 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:43.522460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7980c3-7926-4766-b6ec-64a51eedc1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:43.523108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7980c3-7926-4766-b6ec-64a51eedc1fd service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:43.523417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7980c3-7926-4766-b6ec-64a51eedc1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:43.523717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7980c3-7926-4766-b6ec-64a51eedc1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:43.533040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7980c3-7926-4766-b6ec-64a51eedc1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:43.534531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7980c3-7926-4766-b6ec-64a51eedc1fd service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:43.543994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 753/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:45.736883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31feaf-a82f-443b-aa36-337bcf5bd454 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:45.737472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31feaf-a82f-443b-aa36-337bcf5bd454 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:45.737472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31feaf-a82f-443b-aa36-337bcf5bd454 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:45.738890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 738/1491] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 4 05:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:48:45.745266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11457c19-9080-4f9d-86cf-704bab2ab3d5 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:45.745731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11457c19-9080-4f9d-86cf-704bab2ab3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:45.745865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11457c19-9080-4f9d-86cf-704bab2ab3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:45.768537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-11457c19-9080-4f9d-86cf-704bab2ab3d5 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:48:45.769659 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-11457c19-9080-4f9d-86cf-704bab2ab3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:48:45.778472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 754/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 4 05:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:48:45.820022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31feaf-a82f-443b-aa36-337bcf5bd454 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:48:45.834018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31feaf-a82f-443b-aa36-337bcf5bd454 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:48:45.840551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca3c83-9fb9-4999-a048-01485fc30db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:48:45.841491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca3c83-9fb9-4999-a048-01485fc30db3 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:48:45.841859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca3c83-9fb9-4999-a048-01485fc30db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:48:45.842251 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca3c83-9fb9-4999-a048-01485fc30db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:48:45.855808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca3c83-9fb9-4999-a048-01485fc30db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:48:45.857267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca3c83-9fb9-4999-a048-01485fc30db3 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:48:45.867120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 739/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:48:55.763303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=33 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:48:55.862831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=33 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:50:18.625907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d075e6-beed-4924-8fd5-87d1880927b0 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:18.626974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d075e6-beed-4924-8fd5-87d1880927b0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:18.626974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d075e6-beed-4924-8fd5-87d1880927b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:18.647499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d0d075e6-beed-4924-8fd5-87d1880927b0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:50:18.648709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d0d075e6-beed-4924-8fd5-87d1880927b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:50:18.657240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 755/1494] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:50:18.665588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816fea5-fdba-47da-bec6-25478b343376 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:18.666099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816fea5-fdba-47da-bec6-25478b343376 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:18.666099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816fea5-fdba-47da-bec6-25478b343376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:18.684839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-7816fea5-fdba-47da-bec6-25478b343376 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:50:18.686034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7816fea5-fdba-47da-bec6-25478b343376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:50:18.694461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 740/1495] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:50:18.699289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d075e6-beed-4924-8fd5-87d1880927b0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:50:18.709528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d075e6-beed-4924-8fd5-87d1880927b0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:50:18.715096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0649a0c-98d7-4992-98b9-eb68dd975be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:50:18.715801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0649a0c-98d7-4992-98b9-eb68dd975be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:18.716082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0649a0c-98d7-4992-98b9-eb68dd975be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:18.716353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0649a0c-98d7-4992-98b9-eb68dd975be6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:18.717109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0649a0c-98d7-4992-98b9-eb68dd975be6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:50:18.717417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0649a0c-98d7-4992-98b9-eb68dd975be6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:50:18.718463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: WARNING py.warnings [None req-b0649a0c-98d7-4992-98b9-eb68dd975be6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:50:18.718463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: warnings.warn(msg) Jul 04 05:50:18.718463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: Jul 04 05:50:18.719359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0649a0c-98d7-4992-98b9-eb68dd975be6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:50:18.731774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 756/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:50:18.740190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816fea5-fdba-47da-bec6-25478b343376 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:50:18.750006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816fea5-fdba-47da-bec6-25478b343376 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:50:18.755056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175cfd1-0ea5-4bbb-9317-b9fe6114473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:50:18.755750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175cfd1-0ea5-4bbb-9317-b9fe6114473f admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:18.756034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175cfd1-0ea5-4bbb-9317-b9fe6114473f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:18.756314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175cfd1-0ea5-4bbb-9317-b9fe6114473f admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:18.757005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175cfd1-0ea5-4bbb-9317-b9fe6114473f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:50:18.758875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: WARNING py.warnings [None req-9175cfd1-0ea5-4bbb-9317-b9fe6114473f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:50:18.758875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: warnings.warn(msg) Jul 04 05:50:18.758875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: Jul 04 05:50:18.760108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175cfd1-0ea5-4bbb-9317-b9fe6114473f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:50:18.805038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 741/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:50:18 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:50:18.810919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0649a0c-98d7-4992-98b9-eb68dd975be6 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:50:18.825017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0649a0c-98d7-4992-98b9-eb68dd975be6 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:50:18.886631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c2931-d449-4c7d-a587-c755768869f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:50:18.887336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c2931-d449-4c7d-a587-c755768869f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:18.887623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c2931-d449-4c7d-a587-c755768869f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:18.887900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c2931-d449-4c7d-a587-c755768869f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:18.888612 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c2931-d449-4c7d-a587-c755768869f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:50:18.889645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: WARNING py.warnings [None req-e00c2931-d449-4c7d-a587-c755768869f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:50:18.889645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: warnings.warn(msg) Jul 04 05:50:18.889645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: Jul 04 05:50:18.890846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c2931-d449-4c7d-a587-c755768869f0 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:50:18.913095 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 757/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:50:18 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:50:18.917350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175cfd1-0ea5-4bbb-9317-b9fe6114473f admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:50:18.928683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175cfd1-0ea5-4bbb-9317-b9fe6114473f admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:50:18.933993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566f6c1-8194-4d8a-8071-c710a2a895e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:50:18.934699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566f6c1-8194-4d8a-8071-c710a2a895e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:18.934987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566f6c1-8194-4d8a-8071-c710a2a895e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:18.935288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566f6c1-8194-4d8a-8071-c710a2a895e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:18.936033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566f6c1-8194-4d8a-8071-c710a2a895e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:50:18.937053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: WARNING py.warnings [None req-b566f6c1-8194-4d8a-8071-c710a2a895e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:50:18.937053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: warnings.warn(msg) Jul 04 05:50:18.937053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: Jul 04 05:50:18.937939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566f6c1-8194-4d8a-8071-c710a2a895e9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:50:18.951026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 742/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:50:18.956921 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c2931-d449-4c7d-a587-c755768869f0 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:50:18.972083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c2931-d449-4c7d-a587-c755768869f0 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:50:18.978831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d22b4-4dc6-4889-9f9f-3b1492c209d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:50:18.979523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d22b4-4dc6-4889-9f9f-3b1492c209d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:18.979812 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d22b4-4dc6-4889-9f9f-3b1492c209d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:18.980083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d22b4-4dc6-4889-9f9f-3b1492c209d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:18.980810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d22b4-4dc6-4889-9f9f-3b1492c209d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:50:18.981839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: WARNING py.warnings [None req-137d22b4-4dc6-4889-9f9f-3b1492c209d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:50:18.981839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: warnings.warn(msg) Jul 04 05:50:18.981839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: Jul 04 05:50:18.982715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d22b4-4dc6-4889-9f9f-3b1492c209d1 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:50:18.996304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 758/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:50:19.000852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566f6c1-8194-4d8a-8071-c710a2a895e9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:50:19.011313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566f6c1-8194-4d8a-8071-c710a2a895e9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:50:19.016274 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167851b9-2169-450c-9dd7-5f5bfb8f0758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:50:19.017166 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167851b9-2169-450c-9dd7-5f5bfb8f0758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:19.017448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167851b9-2169-450c-9dd7-5f5bfb8f0758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:19.017750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167851b9-2169-450c-9dd7-5f5bfb8f0758 admin admin] PATH_INFO: `/v3/projects/518a0e2da94a4ff09b2be5e2d4d70465/users/0051f2c1378445ffa5a593c1fbfc9553/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:19.032648 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167851b9-2169-450c-9dd7-5f5bfb8f0758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518a0e2da94a4ff09b2be5e2d4d70465, user_id=0051f2c1378445ffa5a593c1fbfc9553, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:50:19.033678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: WARNING py.warnings [None req-167851b9-2169-450c-9dd7-5f5bfb8f0758 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:50:19.033678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: warnings.warn(msg) Jul 04 05:50:19.033678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: Jul 04 05:50:19.035655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167851b9-2169-450c-9dd7-5f5bfb8f0758 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:50:19.051422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 743/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:50:18 2022] PUT /identity/v3/projects/518a0e2da94a4ff09b2be5e2d4d70465/users/0051f2c1378445ffa5a593c1fbfc9553/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:50:19.064062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d43c6-729c-44a5-b108-d9dd7169f784 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:19.064289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d43c6-729c-44a5-b108-d9dd7169f784 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:19.064590 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d43c6-729c-44a5-b108-d9dd7169f784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:19.092776 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ea3d43c6-729c-44a5-b108-d9dd7169f784 None None] MFA Rules not processed for user `0051f2c1378445ffa5a593c1fbfc9553`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:50:19.093999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ea3d43c6-729c-44a5-b108-d9dd7169f784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:50:19.158681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 759/1502] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:50:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:50:19.173246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5491c-8e9b-49ce-88aa-d3b40903f92e None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:19.173870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5491c-8e9b-49ce-88aa-d3b40903f92e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:19.174314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5491c-8e9b-49ce-88aa-d3b40903f92e None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:19.176525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 744/1503] 10.222.0.42 () {54 vars in 963 bytes} [Mon Jul 4 05:50:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:50:19.183346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81662759-112c-46c5-8958-e142a79425ad None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:19.183628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81662759-112c-46c5-8958-e142a79425ad None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:19.183896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81662759-112c-46c5-8958-e142a79425ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:19.203166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-81662759-112c-46c5-8958-e142a79425ad None None] MFA Rules not processed for user `0f63e110260e4471b0a54576d3573583`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:50:19.204431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-81662759-112c-46c5-8958-e142a79425ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:50:19.253851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:50:19.263706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 760/1504] 10.222.0.42 () {58 vars in 1060 bytes} [Mon Jul 4 05:50:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:50:19.271637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5491c-8e9b-49ce-88aa-d3b40903f92e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:50:19.284420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5491c-8e9b-49ce-88aa-d3b40903f92e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:50:19.288478 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b857-b25c-4a05-b929-b866aa0b83df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:50:19.289119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b857-b25c-4a05-b929-b866aa0b83df service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:19.289396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b857-b25c-4a05-b929-b866aa0b83df service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:19.289668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b857-b25c-4a05-b929-b866aa0b83df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:19.298195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3b857-b25c-4a05-b929-b866aa0b83df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:50:19.299661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3b857-b25c-4a05-b929-b866aa0b83df service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:50:19.309083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 745/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:50:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:50:19.352822 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b368303-0d03-452f-bc88-9adbd2f42411 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:19.353109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b368303-0d03-452f-bc88-9adbd2f42411 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:19.353387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b368303-0d03-452f-bc88-9adbd2f42411 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:19.354682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 761/1506] 10.222.0.42 () {52 vars in 829 bytes} [Mon Jul 4 05:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:50:19.359185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b857-b25c-4a05-b929-b866aa0b83df service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:50:19.427962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b857-b25c-4a05-b929-b866aa0b83df service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:50:19.433558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:50:19.434421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:19.434720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:19.434981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] PATH_INFO: `/v3/users/0051f2c1378445ffa5a593c1fbfc9553` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:19.436289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0051f2c1378445ffa5a593c1fbfc9553)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:50:19.437313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: WARNING py.warnings [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:50:19.437313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: warnings.warn(msg) Jul 04 05:50:19.437313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: Jul 04 05:50:19.438537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:50:19.507104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0051f2c1378445ffa5a593c1fbfc9553'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:50:19.522246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0051f2c1378445ffa5a593c1fbfc9553'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:50:19.535183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0051f2c1378445ffa5a593c1fbfc9553'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:50:19.535955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0051f2c1378445ffa5a593c1fbfc9553'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:50:19.536639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0051f2c1378445ffa5a593c1fbfc9553'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:50:19.553620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:50:19.563741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 746/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:50:19 2022] DELETE /identity/v3/users/0051f2c1378445ffa5a593c1fbfc9553 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:50:19.568522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad8fa3-e233-45ce-88d8-15f85cbf5d70 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:19.568754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad8fa3-e233-45ce-88d8-15f85cbf5d70 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:19.569031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad8fa3-e233-45ce-88d8-15f85cbf5d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:19.587745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a2ad8fa3-e233-45ce-88d8-15f85cbf5d70 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:50:19.588958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a2ad8fa3-e233-45ce-88d8-15f85cbf5d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:50:19.651888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 762/1508] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:50:19.665336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:50:19.666959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:50:19.729407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7136405-6adb-4dab-bb4e-5880e998f76a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:50:19.733973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c754a51-a4ec-4675-a24e-110396bb023b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:50:19.734627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c754a51-a4ec-4675-a24e-110396bb023b service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:19.734910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c754a51-a4ec-4675-a24e-110396bb023b service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:19.735177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c754a51-a4ec-4675-a24e-110396bb023b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:19.746066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c754a51-a4ec-4675-a24e-110396bb023b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:50:19.747554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c754a51-a4ec-4675-a24e-110396bb023b service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:50:19.757535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 747/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:50:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:50:20.092572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad8fa3-e233-45ce-88d8-15f85cbf5d70 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:50:20.094105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a2ad8fa3-e233-45ce-88d8-15f85cbf5d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:50:20.107524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad8fa3-e233-45ce-88d8-15f85cbf5d70 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:50:20.114526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:50:20.115384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:50:20.115665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:50:20.115935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] PATH_INFO: `/v3/projects/518a0e2da94a4ff09b2be5e2d4d70465` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:50:20.117605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] RBAC: Authorizing `identity:delete_project(project_id=518a0e2da94a4ff09b2be5e2d4d70465)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:50:20.118736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: WARNING py.warnings [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:50:20.118736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: warnings.warn(msg) Jul 04 05:50:20.118736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: Jul 04 05:50:20.120027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:50:20.185231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518a0e2da94a4ff09b2be5e2d4d70465', 'request_id': 'req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:50:20.198395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518a0e2da94a4ff09b2be5e2d4d70465', 'request_id': 'req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:50:20.204820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518a0e2da94a4ff09b2be5e2d4d70465', 'request_id': 'req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:50:20.220908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] The token cache is being invalidate because project 518a0e2da94a4ff09b2be5e2d4d70465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:50:20.221189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:50:20.221874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f8f7b44b-460a-47bf-84c3-33e8cc3c37af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:50:20.223564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 763/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:50:20 2022] DELETE /identity/v3/projects/518a0e2da94a4ff09b2be5e2d4d70465 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:50:29.752846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:50:30.217688 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:51:56.879679 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5c610-7391-4651-991d-d11dc0ab48a5 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:51:56.880321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5c610-7391-4651-991d-d11dc0ab48a5 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:51:56.880321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5c610-7391-4651-991d-d11dc0ab48a5 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:51:56.881643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:51:56.887738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5fc36-f4dd-465e-83f3-382691b90517 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:51:56.888526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5fc36-f4dd-465e-83f3-382691b90517 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:51:56.888709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5fc36-f4dd-465e-83f3-382691b90517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:51:56.915451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d8a5fc36-f4dd-465e-83f3-382691b90517 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:51:56.916765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d8a5fc36-f4dd-465e-83f3-382691b90517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:51:56.980208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:51:57.136954 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5c610-7391-4651-991d-d11dc0ab48a5 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:51:57.139049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1dc5c610-7391-4651-991d-d11dc0ab48a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:51:57.223232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5c610-7391-4651-991d-d11dc0ab48a5 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:51:57.229521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40a9b9-c82c-40ef-8b2c-cc695847c2d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:51:57.230188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40a9b9-c82c-40ef-8b2c-cc695847c2d6 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:51:57.230471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40a9b9-c82c-40ef-8b2c-cc695847c2d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:51:57.230726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40a9b9-c82c-40ef-8b2c-cc695847c2d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:51:57.239265 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40a9b9-c82c-40ef-8b2c-cc695847c2d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:51:57.240883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40a9b9-c82c-40ef-8b2c-cc695847c2d6 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:51:57.250383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 749/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:51:58.088299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6a419-2fd1-4eea-b06d-b1a070ba1eab None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:51:58.089151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6a419-2fd1-4eea-b06d-b1a070ba1eab None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:51:58.089151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6a419-2fd1-4eea-b06d-b1a070ba1eab None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:51:58.091319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:51:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:51:58.097445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e72c3-80da-47a1-be19-e2a99e388a74 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:51:58.098149 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e72c3-80da-47a1-be19-e2a99e388a74 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:51:58.098357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e72c3-80da-47a1-be19-e2a99e388a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:51:58.123396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-1d8e72c3-80da-47a1-be19-e2a99e388a74 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:51:58.124609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1d8e72c3-80da-47a1-be19-e2a99e388a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:51:58.133783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:51:58.296429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6a419-2fd1-4eea-b06d-b1a070ba1eab None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:51:58.318558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6a419-2fd1-4eea-b06d-b1a070ba1eab None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:51:58.325568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5be5a-6a7c-4a01-9f32-42be6c742ec9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:51:58.326223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5be5a-6a7c-4a01-9f32-42be6c742ec9 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:51:58.326505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5be5a-6a7c-4a01-9f32-42be6c742ec9 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:51:58.326779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5be5a-6a7c-4a01-9f32-42be6c742ec9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:51:58.335038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5be5a-6a7c-4a01-9f32-42be6c742ec9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:51:58.336691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5be5a-6a7c-4a01-9f32-42be6c742ec9 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:51:58.346989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 766/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:51:58.938373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e748d-99f2-4198-b16f-62ccb60c0bb7 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:51:58.938663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e748d-99f2-4198-b16f-62ccb60c0bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:51:58.938972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e748d-99f2-4198-b16f-62ccb60c0bb7 None None] PATH_INFO: `/v3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:51:58.940370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 751/1517] 10.222.0.71 () {54 vars in 925 bytes} [Mon Jul 4 05:51:58 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:51:58.948164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9065eb-f25d-40ae-bb49-4fdd74334ac7 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:51:58.948489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9065eb-f25d-40ae-bb49-4fdd74334ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:51:58.948810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9065eb-f25d-40ae-bb49-4fdd74334ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:51:58.976060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-da9065eb-f25d-40ae-bb49-4fdd74334ac7 None None] MFA Rules not processed for user `486de932d1db422ba2c3d84142f9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:51:58.977301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-da9065eb-f25d-40ae-bb49-4fdd74334ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:51:59.043090 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 767/1518] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Jul 4 05:51:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:52:02.870945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5037fa2-dc9f-4f45-b8f9-22c960577d0e None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:02.871554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5037fa2-dc9f-4f45-b8f9-22c960577d0e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:02.871554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5037fa2-dc9f-4f45-b8f9-22c960577d0e None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:02.872849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 752/1519] 10.222.0.42 () {54 vars in 960 bytes} [Mon Jul 4 05:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:52:02.878242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed798f-d405-4fef-aae7-e5b24237f1d4 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:02.878689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed798f-d405-4fef-aae7-e5b24237f1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:02.878793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed798f-d405-4fef-aae7-e5b24237f1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:02.897831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-38ed798f-d405-4fef-aae7-e5b24237f1d4 None None] MFA Rules not processed for user `486de932d1db422ba2c3d84142f9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:52:02.899091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-38ed798f-d405-4fef-aae7-e5b24237f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:02.909181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 768/1520] 10.222.0.42 () {58 vars in 1057 bytes} [Mon Jul 4 05:52:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:52:02.917130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5037fa2-dc9f-4f45-b8f9-22c960577d0e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:52:02.932007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5037fa2-dc9f-4f45-b8f9-22c960577d0e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:52:02.937515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8684c-672b-4a24-be8a-cc875fbe78ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:52:02.938167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8684c-672b-4a24-be8a-cc875fbe78ff service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:02.938442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8684c-672b-4a24-be8a-cc875fbe78ff service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:02.938687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8684c-672b-4a24-be8a-cc875fbe78ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:02.947270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8684c-672b-4a24-be8a-cc875fbe78ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:52:02.948896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8684c-672b-4a24-be8a-cc875fbe78ff service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:52:02.958725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 753/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:52:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:52:09.584107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63c64b-5a4b-4b40-9e56-2f666b6acfc3 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:09.584688 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63c64b-5a4b-4b40-9e56-2f666b6acfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:09.584688 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63c64b-5a4b-4b40-9e56-2f666b6acfc3 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:09.585944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:52:09.591821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb06be-9236-4eb7-be55-2dd4a6aacf1a None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:09.592275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb06be-9236-4eb7-be55-2dd4a6aacf1a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:09.592378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb06be-9236-4eb7-be55-2dd4a6aacf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:09.619950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-3bcb06be-9236-4eb7-be55-2dd4a6aacf1a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:52:09.621152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3bcb06be-9236-4eb7-be55-2dd4a6aacf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:09.630221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:52:09.798603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63c64b-5a4b-4b40-9e56-2f666b6acfc3 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:52:09.821431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63c64b-5a4b-4b40-9e56-2f666b6acfc3 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:52:09.828003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9d715-bf5b-45ed-87b6-8b0b080e3389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:52:09.828651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9d715-bf5b-45ed-87b6-8b0b080e3389 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:09.828933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9d715-bf5b-45ed-87b6-8b0b080e3389 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:09.829206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9d715-bf5b-45ed-87b6-8b0b080e3389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:09.837530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9d715-bf5b-45ed-87b6-8b0b080e3389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:52:09.839094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9d715-bf5b-45ed-87b6-8b0b080e3389 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:52:09.848604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 770/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:52:10.676803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48017d20-d4fd-4b0b-91b7-ba9157b47df1 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:10.677708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48017d20-d4fd-4b0b-91b7-ba9157b47df1 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:10.677708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48017d20-d4fd-4b0b-91b7-ba9157b47df1 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:10.680370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 05:52:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:52:10.688724 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fb7b4-4c5b-4397-9659-c884ca9e6f93 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:10.689163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fb7b4-4c5b-4397-9659-c884ca9e6f93 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:10.689370 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fb7b4-4c5b-4397-9659-c884ca9e6f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:10.711603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-359fb7b4-4c5b-4397-9659-c884ca9e6f93 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:52:10.712873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-359fb7b4-4c5b-4397-9659-c884ca9e6f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:10.723062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 05:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:52:10.883062 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48017d20-d4fd-4b0b-91b7-ba9157b47df1 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:52:10.902334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48017d20-d4fd-4b0b-91b7-ba9157b47df1 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:52:10.908340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc08d9-be06-4d41-90d2-7222c1567970 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:52:10.908998 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc08d9-be06-4d41-90d2-7222c1567970 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:10.909284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc08d9-be06-4d41-90d2-7222c1567970 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:10.909546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc08d9-be06-4d41-90d2-7222c1567970 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:10.918075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc08d9-be06-4d41-90d2-7222c1567970 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:52:10.919774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc08d9-be06-4d41-90d2-7222c1567970 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:52:10.929849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 756/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:52:14.781713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e860c-3e16-4707-af0f-f212a6af965d None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:14.782282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e860c-3e16-4707-af0f-f212a6af965d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:14.782282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e860c-3e16-4707-af0f-f212a6af965d None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:14.783682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:52:14.790097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b532b-0ec4-4f98-a313-497250ec99ae None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:14.790854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b532b-0ec4-4f98-a313-497250ec99ae None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:14.791078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b532b-0ec4-4f98-a313-497250ec99ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:14.816714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-b12b532b-0ec4-4f98-a313-497250ec99ae None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:52:14.817927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b12b532b-0ec4-4f98-a313-497250ec99ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:14.826701 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:52:14.915026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e860c-3e16-4707-af0f-f212a6af965d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:52:14.917900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-887e860c-3e16-4707-af0f-f212a6af965d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:14.983226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e860c-3e16-4707-af0f-f212a6af965d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:52:14.987587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fe06a-c9db-4651-ab6a-61745980bf04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:52:14.988230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fe06a-c9db-4651-ab6a-61745980bf04 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:14.988505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fe06a-c9db-4651-ab6a-61745980bf04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:14.988780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fe06a-c9db-4651-ab6a-61745980bf04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:15.005461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fe06a-c9db-4651-ab6a-61745980bf04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:52:15.007689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fe06a-c9db-4651-ab6a-61745980bf04 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:52:15.014060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:52:15.019516 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 773/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:52:17.046544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364ecf6-4ddb-498d-abdd-04529f850f63 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:17.047168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364ecf6-4ddb-498d-abdd-04529f850f63 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:17.047168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364ecf6-4ddb-498d-abdd-04529f850f63 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:17.048538 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:52:17.055263 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4948d57-5fc9-4a46-a92c-75e013ff867f None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:17.056005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4948d57-5fc9-4a46-a92c-75e013ff867f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:17.056153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4948d57-5fc9-4a46-a92c-75e013ff867f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:17.076914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c4948d57-5fc9-4a46-a92c-75e013ff867f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:52:17.078147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c4948d57-5fc9-4a46-a92c-75e013ff867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:17.087494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:52:17.124391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364ecf6-4ddb-498d-abdd-04529f850f63 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:52:17.127567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d364ecf6-4ddb-498d-abdd-04529f850f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:17.145024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364ecf6-4ddb-498d-abdd-04529f850f63 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:52:17.149021 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ff6bb-a6c6-44e3-bb85-7e6f401daeb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:52:17.149660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff6bb-a6c6-44e3-bb85-7e6f401daeb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:17.149937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff6bb-a6c6-44e3-bb85-7e6f401daeb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:17.150224 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff6bb-a6c6-44e3-bb85-7e6f401daeb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:17.159560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ff6bb-a6c6-44e3-bb85-7e6f401daeb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:52:17.160990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ff6bb-a6c6-44e3-bb85-7e6f401daeb5 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:52:17.170808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 759/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:52:20.750454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05273863-2b60-45e9-85f9-97885d66a1ad None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:20.751031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05273863-2b60-45e9-85f9-97885d66a1ad None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:20.751031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05273863-2b60-45e9-85f9-97885d66a1ad None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:20.752468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:52:20.757943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa85f19-73dd-4360-9bea-ee4554fef95c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:20.758399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa85f19-73dd-4360-9bea-ee4554fef95c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:20.758525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa85f19-73dd-4360-9bea-ee4554fef95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:20.765646 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=62 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:52:20.779331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4aa85f19-73dd-4360-9bea-ee4554fef95c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:52:20.780423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4aa85f19-73dd-4360-9bea-ee4554fef95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:20.789907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:52:21.089856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05273863-2b60-45e9-85f9-97885d66a1ad None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:52:21.091997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-05273863-2b60-45e9-85f9-97885d66a1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:21.152647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05273863-2b60-45e9-85f9-97885d66a1ad None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:52:21.156690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7248e-184b-4c2b-8121-716f4274bdd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:52:21.157330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7248e-184b-4c2b-8121-716f4274bdd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:21.157609 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7248e-184b-4c2b-8121-716f4274bdd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:21.157880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7248e-184b-4c2b-8121-716f4274bdd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:21.167572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7248e-184b-4c2b-8121-716f4274bdd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:52:21.169060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7248e-184b-4c2b-8121-716f4274bdd9 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:52:21.178586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 776/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:52:23.239081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d073e-ec5a-4385-90df-59f009db9483 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:23.239750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d073e-ec5a-4385-90df-59f009db9483 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:23.239750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d073e-ec5a-4385-90df-59f009db9483 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:23.241089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 05:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:52:23.247480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56792588-b6fd-47fb-ab84-72723beef4fd None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:23.248208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56792588-b6fd-47fb-ab84-72723beef4fd None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:23.248390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56792588-b6fd-47fb-ab84-72723beef4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:23.272695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-56792588-b6fd-47fb-ab84-72723beef4fd None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:52:23.273915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-56792588-b6fd-47fb-ab84-72723beef4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:23.283702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 05:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:52:23.585397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d073e-ec5a-4385-90df-59f009db9483 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:52:23.588642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-021d073e-ec5a-4385-90df-59f009db9483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:52:23.604193 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d073e-ec5a-4385-90df-59f009db9483 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:52:23.608294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b6f30-4351-4e32-b7c1-edf124e34bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:52:23.608947 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b6f30-4351-4e32-b7c1-edf124e34bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:52:23.609228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b6f30-4351-4e32-b7c1-edf124e34bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:52:23.609501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b6f30-4351-4e32-b7c1-edf124e34bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:52:23.619482 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b6f30-4351-4e32-b7c1-edf124e34bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:52:23.620930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b6f30-4351-4e32-b7c1-edf124e34bf8 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:52:23.630788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 762/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:52:33.267064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=32 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:52:33.626389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=8 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:51.604598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed005ea-ceba-496d-ad9b-1bfa956a09f9 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:51.605357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed005ea-ceba-496d-ad9b-1bfa956a09f9 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:51.605357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed005ea-ceba-496d-ad9b-1bfa956a09f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:51.629209 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-3ed005ea-ceba-496d-ad9b-1bfa956a09f9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:51.630443 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3ed005ea-ceba-496d-ad9b-1bfa956a09f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:51.644244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 778/1540] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:51.654254 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dc1a1-49bb-4c73-9a17-efc93b8dfce2 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:51.656284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dc1a1-49bb-4c73-9a17-efc93b8dfce2 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:51.656284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dc1a1-49bb-4c73-9a17-efc93b8dfce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:51.688051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-bc7dc1a1-49bb-4c73-9a17-efc93b8dfce2 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:51.689291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-bc7dc1a1-49bb-4c73-9a17-efc93b8dfce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:51.702041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 763/1541] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:51.706893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed005ea-ceba-496d-ad9b-1bfa956a09f9 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:51.717251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed005ea-ceba-496d-ad9b-1bfa956a09f9 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:51.727667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f2d8a-8db0-40a2-ba09-2b02a24f6b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:51.728362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f2d8a-8db0-40a2-ba09-2b02a24f6b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:51.728643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f2d8a-8db0-40a2-ba09-2b02a24f6b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:51.728924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f2d8a-8db0-40a2-ba09-2b02a24f6b66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:51.729665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f2d8a-8db0-40a2-ba09-2b02a24f6b66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:53:51.729973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f2d8a-8db0-40a2-ba09-2b02a24f6b66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:51.731651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f2d8a-8db0-40a2-ba09-2b02a24f6b66 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:51.738396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da6f2a-c7db-40a7-8a7b-41b58d59a0c5 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:51.738685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da6f2a-c7db-40a7-8a7b-41b58d59a0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:51.738969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da6f2a-c7db-40a7-8a7b-41b58d59a0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:51.770045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 779/1542] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:53:51.771054 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-38da6f2a-c7db-40a7-8a7b-41b58d59a0c5 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:51.772315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-38da6f2a-c7db-40a7-8a7b-41b58d59a0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:51.781495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 764/1543] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 04 05:53:51.785904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f2d8a-8db0-40a2-ba09-2b02a24f6b66 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:51.796435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f2d8a-8db0-40a2-ba09-2b02a24f6b66 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:51.796689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694ce1-df61-4245-8c4b-65811600bee3 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:51.796947 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694ce1-df61-4245-8c4b-65811600bee3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:51.797232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694ce1-df61-4245-8c4b-65811600bee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:51.816208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4ca7-3de6-4eb9-bdb4-e229a54d9e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:51.816899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4ca7-3de6-4eb9-bdb4-e229a54d9e6e admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:51.817185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4ca7-3de6-4eb9-bdb4-e229a54d9e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:51.817463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4ca7-3de6-4eb9-bdb4-e229a54d9e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:51.818169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b4ca7-3de6-4eb9-bdb4-e229a54d9e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:51.819207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: WARNING py.warnings [None req-3d6b4ca7-3de6-4eb9-bdb4-e229a54d9e6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:53:51.819207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: warnings.warn(msg) Jul 04 05:53:51.819207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: Jul 04 05:53:51.821416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b4ca7-3de6-4eb9-bdb4-e229a54d9e6e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:51.837808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-05694ce1-df61-4245-8c4b-65811600bee3 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:51.839044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-05694ce1-df61-4245-8c4b-65811600bee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:51.864186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 765/1544] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:51.865828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5300ded-556d-437d-a63b-77887f52d925 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:51.866209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5300ded-556d-437d-a63b-77887f52d925 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:51.866491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5300ded-556d-437d-a63b-77887f52d925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:51.878043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 780/1545] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:53:51 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:51.879415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f080d1b-35c9-4e3e-aa1e-d09cc6ece892 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:51.879704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f080d1b-35c9-4e3e-aa1e-d09cc6ece892 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:51.879982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f080d1b-35c9-4e3e-aa1e-d09cc6ece892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:51.898441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a5300ded-556d-437d-a63b-77887f52d925 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:51.899597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a5300ded-556d-437d-a63b-77887f52d925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:51.927829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-3f080d1b-35c9-4e3e-aa1e-d09cc6ece892 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:51.929056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3f080d1b-35c9-4e3e-aa1e-d09cc6ece892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.024223 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 766/1546] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.025472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5300ded-556d-437d-a63b-77887f52d925 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.036401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5300ded-556d-437d-a63b-77887f52d925 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.041011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33018db-4989-4338-ba37-a6252fc49731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.041684 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33018db-4989-4338-ba37-a6252fc49731 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.041965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33018db-4989-4338-ba37-a6252fc49731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.042572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33018db-4989-4338-ba37-a6252fc49731 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.042967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33018db-4989-4338-ba37-a6252fc49731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:53:52.043292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33018db-4989-4338-ba37-a6252fc49731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.044324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: WARNING py.warnings [None req-a33018db-4989-4338-ba37-a6252fc49731 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:53:52.044324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: warnings.warn(msg) Jul 04 05:53:52.044324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: Jul 04 05:53:52.045226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33018db-4989-4338-ba37-a6252fc49731 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.045341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 781/1547] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.046145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f080d1b-35c9-4e3e-aa1e-d09cc6ece892 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.056293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f080d1b-35c9-4e3e-aa1e-d09cc6ece892 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.058703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 767/1548] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:53:52.060012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eec4d6-404a-4465-92fa-7d5a8a6ebac8 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.060297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eec4d6-404a-4465-92fa-7d5a8a6ebac8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.060570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eec4d6-404a-4465-92fa-7d5a8a6ebac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.061676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b777a7-7bbe-4963-8117-2a4b0ade3412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.062414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b777a7-7bbe-4963-8117-2a4b0ade3412 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.062698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b777a7-7bbe-4963-8117-2a4b0ade3412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.062971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b777a7-7bbe-4963-8117-2a4b0ade3412 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.070128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b777a7-7bbe-4963-8117-2a4b0ade3412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.072581 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b777a7-7bbe-4963-8117-2a4b0ade3412 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.078519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-41eec4d6-404a-4465-92fa-7d5a8a6ebac8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:52.079767 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-41eec4d6-404a-4465-92fa-7d5a8a6ebac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.088022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 768/1549] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.089507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302baa1-e840-4037-a507-63c988e026ee None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.089830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302baa1-e840-4037-a507-63c988e026ee None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.090116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302baa1-e840-4037-a507-63c988e026ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.102074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 782/1550] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/users => generated 289 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.103676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54d42e-4298-48e9-a623-0ce25ef81fae None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.103993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54d42e-4298-48e9-a623-0ce25ef81fae None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.104277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54d42e-4298-48e9-a623-0ce25ef81fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.112891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2302baa1-e840-4037-a507-63c988e026ee None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:52.114122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2302baa1-e840-4037-a507-63c988e026ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.122465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-9d54d42e-4298-48e9-a623-0ce25ef81fae None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:52.122914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 769/1551] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.124638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9d54d42e-4298-48e9-a623-0ce25ef81fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.124877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1addef-103c-4ef8-9bd1-658a3c5ea0c9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.124877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1addef-103c-4ef8-9bd1-658a3c5ea0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.124877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1addef-103c-4ef8-9bd1-658a3c5ea0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.134134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 783/1552] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.135472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f29b-a6d1-4173-8889-e66efe0a94b8 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.135757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f29b-a6d1-4173-8889-e66efe0a94b8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.136060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f29b-a6d1-4173-8889-e66efe0a94b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.144628 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-3e1addef-103c-4ef8-9bd1-658a3c5ea0c9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:52.145836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3e1addef-103c-4ef8-9bd1-658a3c5ea0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.153673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 770/1553] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.154831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac8aa-2a5a-4b64-a3b7-706bf2053a47 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.155114 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac8aa-2a5a-4b64-a3b7-706bf2053a47 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.155423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac8aa-2a5a-4b64-a3b7-706bf2053a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.158202 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-2565f29b-a6d1-4173-8889-e66efe0a94b8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:52.159450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2565f29b-a6d1-4173-8889-e66efe0a94b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.167640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 784/1554] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.170870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565f29b-a6d1-4173-8889-e66efe0a94b8 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.173992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-805ac8aa-2a5a-4b64-a3b7-706bf2053a47 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:52.175164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-805ac8aa-2a5a-4b64-a3b7-706bf2053a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.180916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565f29b-a6d1-4173-8889-e66efe0a94b8 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.183837 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 771/1555] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.184985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56550c39-8844-4067-be3e-9047cac509a2 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.185305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56550c39-8844-4067-be3e-9047cac509a2 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.185551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56550c39-8844-4067-be3e-9047cac509a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.190482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690446e-9434-40bd-bcbc-f488e0c45acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.191149 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690446e-9434-40bd-bcbc-f488e0c45acd admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.191452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690446e-9434-40bd-bcbc-f488e0c45acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.191726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690446e-9434-40bd-bcbc-f488e0c45acd admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.192410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690446e-9434-40bd-bcbc-f488e0c45acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.194474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690446e-9434-40bd-bcbc-f488e0c45acd admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.204993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-56550c39-8844-4067-be3e-9047cac509a2 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:52.206279 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-56550c39-8844-4067-be3e-9047cac509a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.215074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 772/1556] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.215898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56550c39-8844-4067-be3e-9047cac509a2 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.222190 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 785/1557] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/projects => generated 357 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.223500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2916ac-f2d8-49c2-8ae1-69fa3a9a108e None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.223791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2916ac-f2d8-49c2-8ae1-69fa3a9a108e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.224062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2916ac-f2d8-49c2-8ae1-69fa3a9a108e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.225264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56550c39-8844-4067-be3e-9047cac509a2 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.241478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-1d2916ac-f2d8-49c2-8ae1-69fa3a9a108e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:52.242660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1d2916ac-f2d8-49c2-8ae1-69fa3a9a108e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.282528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b8965-0042-4e76-90e1-e0b92a3a64da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.283268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b8965-0042-4e76-90e1-e0b92a3a64da admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.283573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b8965-0042-4e76-90e1-e0b92a3a64da admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.283844 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b8965-0042-4e76-90e1-e0b92a3a64da admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.284584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b8965-0042-4e76-90e1-e0b92a3a64da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.286306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b8965-0042-4e76-90e1-e0b92a3a64da admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.296890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 773/1558] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:52.300116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a767c73-0ddc-4526-aa11-194b288754cd None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.300116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a767c73-0ddc-4526-aa11-194b288754cd None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.300116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a767c73-0ddc-4526-aa11-194b288754cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.308376 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 786/1559] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.309365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2916ac-f2d8-49c2-8ae1-69fa3a9a108e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.319140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2916ac-f2d8-49c2-8ae1-69fa3a9a108e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.320829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-5a767c73-0ddc-4526-aa11-194b288754cd None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:52.322050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5a767c73-0ddc-4526-aa11-194b288754cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.323823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca12aa0-1304-4a3b-bcf6-113da4a487a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.324478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca12aa0-1304-4a3b-bcf6-113da4a487a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.324759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca12aa0-1304-4a3b-bcf6-113da4a487a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.325033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca12aa0-1304-4a3b-bcf6-113da4a487a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.325762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca12aa0-1304-4a3b-bcf6-113da4a487a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:53:52.326066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca12aa0-1304-4a3b-bcf6-113da4a487a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.327726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca12aa0-1304-4a3b-bcf6-113da4a487a1 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.329337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:52.330094 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 774/1560] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.331468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07861d83-e2ad-44bc-8a88-68478c095ed8 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.331738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07861d83-e2ad-44bc-8a88-68478c095ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.332012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07861d83-e2ad-44bc-8a88-68478c095ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.345327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 787/1561] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:53:52.346488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca12aa0-1304-4a3b-bcf6-113da4a487a1 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.355865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-07861d83-e2ad-44bc-8a88-68478c095ed8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:52.356223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca12aa0-1304-4a3b-bcf6-113da4a487a1 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.357070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-07861d83-e2ad-44bc-8a88-68478c095ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:52.361081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537921f-26f6-4453-8066-349054c456f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.361721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537921f-26f6-4453-8066-349054c456f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.362005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537921f-26f6-4453-8066-349054c456f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.362282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537921f-26f6-4453-8066-349054c456f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.363008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537921f-26f6-4453-8066-349054c456f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:53:52.363346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537921f-26f6-4453-8066-349054c456f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.364974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537921f-26f6-4453-8066-349054c456f7 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.365194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 775/1562] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:52.368119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07861d83-e2ad-44bc-8a88-68478c095ed8 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.377927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 788/1563] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:53:52.378705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537921f-26f6-4453-8066-349054c456f7 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.381611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07861d83-e2ad-44bc-8a88-68478c095ed8 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.387109 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbb350-32ea-495b-8439-27505cefacc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.387860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb350-32ea-495b-8439-27505cefacc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.388068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537921f-26f6-4453-8066-349054c456f7 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.388187 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb350-32ea-495b-8439-27505cefacc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.388373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb350-32ea-495b-8439-27505cefacc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.389096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbb350-32ea-495b-8439-27505cefacc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:53:52.389408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbb350-32ea-495b-8439-27505cefacc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.391106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbb350-32ea-495b-8439-27505cefacc3 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.392639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1894e-cfe0-4b32-8fdf-ff7c48288531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.393284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1894e-cfe0-4b32-8fdf-ff7c48288531 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.393687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1894e-cfe0-4b32-8fdf-ff7c48288531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.393967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1894e-cfe0-4b32-8fdf-ff7c48288531 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.394650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1894e-cfe0-4b32-8fdf-ff7c48288531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.396593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1894e-cfe0-4b32-8fdf-ff7c48288531 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.403200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 776/1564] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:53:52.404064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbb350-32ea-495b-8439-27505cefacc3 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.411179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:52.416292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbb350-32ea-495b-8439-27505cefacc3 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.421484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a3287-6955-4eda-a165-d56f05fd0dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.422219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a3287-6955-4eda-a165-d56f05fd0dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.422528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a3287-6955-4eda-a165-d56f05fd0dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.422772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a3287-6955-4eda-a165-d56f05fd0dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.423697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a3287-6955-4eda-a165-d56f05fd0dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.425161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 789/1565] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/users => generated 279 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.425469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a3287-6955-4eda-a165-d56f05fd0dfb admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.425995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1894e-cfe0-4b32-8fdf-ff7c48288531 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.436129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1894e-cfe0-4b32-8fdf-ff7c48288531 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.436849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 777/1566] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:52.437674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a3287-6955-4eda-a165-d56f05fd0dfb admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.440917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e232bc3-126a-4190-b16a-572ef85f7237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.441574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e232bc3-126a-4190-b16a-572ef85f7237 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.441858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e232bc3-126a-4190-b16a-572ef85f7237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.442135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e232bc3-126a-4190-b16a-572ef85f7237 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.442860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e232bc3-126a-4190-b16a-572ef85f7237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:53:52.443167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e232bc3-126a-4190-b16a-572ef85f7237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.444830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e232bc3-126a-4190-b16a-572ef85f7237 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.447159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a3287-6955-4eda-a165-d56f05fd0dfb admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.451596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa188d-7ca6-4392-b8be-9d37835c61b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.452229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa188d-7ca6-4392-b8be-9d37835c61b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.452549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa188d-7ca6-4392-b8be-9d37835c61b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.452828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa188d-7ca6-4392-b8be-9d37835c61b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.453587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa188d-7ca6-4392-b8be-9d37835c61b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:53:52.453889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa188d-7ca6-4392-b8be-9d37835c61b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.455484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa188d-7ca6-4392-b8be-9d37835c61b6 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.458174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 790/1567] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:53:52.458954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e232bc3-126a-4190-b16a-572ef85f7237 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.467485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 778/1568] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:53:52.468408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa188d-7ca6-4392-b8be-9d37835c61b6 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.470536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e232bc3-126a-4190-b16a-572ef85f7237 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.476078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fc095-f4dd-47a6-8194-e79749fa67e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.476742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fc095-f4dd-47a6-8194-e79749fa67e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.477028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fc095-f4dd-47a6-8194-e79749fa67e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.477303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fc095-f4dd-47a6-8194-e79749fa67e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.477997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fc095-f4dd-47a6-8194-e79749fa67e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.478470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa188d-7ca6-4392-b8be-9d37835c61b6 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.479966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fc095-f4dd-47a6-8194-e79749fa67e7 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.483015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df201815-906e-476b-9fb6-af463586f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.483761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df201815-906e-476b-9fb6-af463586f16a admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.484184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df201815-906e-476b-9fb6-af463586f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.484542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df201815-906e-476b-9fb6-af463586f16a admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.485321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df201815-906e-476b-9fb6-af463586f16a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:53:52.485694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df201815-906e-476b-9fb6-af463586f16a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.487351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df201815-906e-476b-9fb6-af463586f16a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.499752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 779/1569] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:53:52.500686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df201815-906e-476b-9fb6-af463586f16a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.509878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 791/1570] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.509878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fc095-f4dd-47a6-8194-e79749fa67e7 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.517049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df201815-906e-476b-9fb6-af463586f16a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.571732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fc095-f4dd-47a6-8194-e79749fa67e7 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.575852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314294ae-7c5a-47ef-939c-c80dfb1d6c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.576375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9265b27-1468-4c77-9f26-d26be640b337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.576575 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314294ae-7c5a-47ef-939c-c80dfb1d6c2d admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.576866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314294ae-7c5a-47ef-939c-c80dfb1d6c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.577063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9265b27-1468-4c77-9f26-d26be640b337 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.577186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314294ae-7c5a-47ef-939c-c80dfb1d6c2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.577317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9265b27-1468-4c77-9f26-d26be640b337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.577592 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9265b27-1468-4c77-9f26-d26be640b337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.578317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9265b27-1468-4c77-9f26-d26be640b337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.578430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314294ae-7c5a-47ef-939c-c80dfb1d6c2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.580282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314294ae-7c5a-47ef-939c-c80dfb1d6c2d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.580407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9265b27-1468-4c77-9f26-d26be640b337 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.609255 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 780/1571] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.610432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314294ae-7c5a-47ef-939c-c80dfb1d6c2d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.610548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 792/1572] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.611433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9265b27-1468-4c77-9f26-d26be640b337 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.677400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9265b27-1468-4c77-9f26-d26be640b337 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.682450 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314294ae-7c5a-47ef-939c-c80dfb1d6c2d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.683137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c426acc-b4b6-4678-94cb-7cfcc1bd37f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.684256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c426acc-b4b6-4678-94cb-7cfcc1bd37f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.684533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c426acc-b4b6-4678-94cb-7cfcc1bd37f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.684804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c426acc-b4b6-4678-94cb-7cfcc1bd37f0 admin admin] PATH_INFO: `/v3/projects/413717d499504cb29cb26d08bedf7e0a/users/3e05d19530a74e05a11be774102b425a/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.687203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e99c0-cafe-487b-852d-db4271732dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.687952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e99c0-cafe-487b-852d-db4271732dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.688236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e99c0-cafe-487b-852d-db4271732dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.688519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e99c0-cafe-487b-852d-db4271732dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.689276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e99c0-cafe-487b-852d-db4271732dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.690917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e99c0-cafe-487b-852d-db4271732dcf admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.700465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c426acc-b4b6-4678-94cb-7cfcc1bd37f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=413717d499504cb29cb26d08bedf7e0a, user_id=3e05d19530a74e05a11be774102b425a, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.701528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: WARNING py.warnings [None req-6c426acc-b4b6-4678-94cb-7cfcc1bd37f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:53:52.701528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: warnings.warn(msg) Jul 04 05:53:52.701528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: Jul 04 05:53:52.703687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c426acc-b4b6-4678-94cb-7cfcc1bd37f0 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.706762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:52.710784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 781/1573] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:52.710784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e99c0-cafe-487b-852d-db4271732dcf admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.719359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e99c0-cafe-487b-852d-db4271732dcf admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.721554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 793/1574] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:53:52 2022] PUT /identity/v3/projects/413717d499504cb29cb26d08bedf7e0a/users/3e05d19530a74e05a11be774102b425a/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:52.722387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c426acc-b4b6-4678-94cb-7cfcc1bd37f0 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.763527 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:52.776415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb92ec0-b69b-492d-a103-0852ceef3967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.777096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb92ec0-b69b-492d-a103-0852ceef3967 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.777378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb92ec0-b69b-492d-a103-0852ceef3967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.777652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb92ec0-b69b-492d-a103-0852ceef3967 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.778370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb92ec0-b69b-492d-a103-0852ceef3967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.780376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb92ec0-b69b-492d-a103-0852ceef3967 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.784779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c426acc-b4b6-4678-94cb-7cfcc1bd37f0 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.789916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ad816-52ed-43ae-8f2e-fbe467337010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.790586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ad816-52ed-43ae-8f2e-fbe467337010 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.790870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ad816-52ed-43ae-8f2e-fbe467337010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.791144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ad816-52ed-43ae-8f2e-fbe467337010 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.791871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ad816-52ed-43ae-8f2e-fbe467337010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.793832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ad816-52ed-43ae-8f2e-fbe467337010 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.809971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 782/1575] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.810809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb92ec0-b69b-492d-a103-0852ceef3967 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.823658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 794/1576] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.823658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ad816-52ed-43ae-8f2e-fbe467337010 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.876204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb92ec0-b69b-492d-a103-0852ceef3967 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.885200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ad816-52ed-43ae-8f2e-fbe467337010 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.889843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630c2e0-40d2-4ef6-a9a8-e9e5cb5df434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.890572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630c2e0-40d2-4ef6-a9a8-e9e5cb5df434 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.890856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630c2e0-40d2-4ef6-a9a8-e9e5cb5df434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.891131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630c2e0-40d2-4ef6-a9a8-e9e5cb5df434 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.891867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630c2e0-40d2-4ef6-a9a8-e9e5cb5df434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.893820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630c2e0-40d2-4ef6-a9a8-e9e5cb5df434 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.918384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 795/1577] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.919500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630c2e0-40d2-4ef6-a9a8-e9e5cb5df434 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.929660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630c2e0-40d2-4ef6-a9a8-e9e5cb5df434 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.934353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce66b8-20b6-44dd-ada4-31ef0be513cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.935094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce66b8-20b6-44dd-ada4-31ef0be513cf admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.935323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce66b8-20b6-44dd-ada4-31ef0be513cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.935622 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce66b8-20b6-44dd-ada4-31ef0be513cf admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.936321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce66b8-20b6-44dd-ada4-31ef0be513cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.938243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce66b8-20b6-44dd-ada4-31ef0be513cf admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.939064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33b2b9-ddf1-4afe-9901-3d9c4b58c9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.939768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33b2b9-ddf1-4afe-9901-3d9c4b58c9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.940050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33b2b9-ddf1-4afe-9901-3d9c4b58c9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.940319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33b2b9-ddf1-4afe-9901-3d9c4b58c9c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.941009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33b2b9-ddf1-4afe-9901-3d9c4b58c9c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.942893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33b2b9-ddf1-4afe-9901-3d9c4b58c9c9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:52.960780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 796/1578] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.961650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce66b8-20b6-44dd-ada4-31ef0be513cf admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.971925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce66b8-20b6-44dd-ada4-31ef0be513cf admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:52.975238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 783/1579] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:52.976083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33b2b9-ddf1-4afe-9901-3d9c4b58c9c9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:52.976833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868415b-7301-4edc-9e09-34958e654886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:52.977498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868415b-7301-4edc-9e09-34958e654886 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:52.977777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868415b-7301-4edc-9e09-34958e654886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:52.978082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868415b-7301-4edc-9e09-34958e654886 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:52.978904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868415b-7301-4edc-9e09-34958e654886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:52.980923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868415b-7301-4edc-9e09-34958e654886 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.004247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 797/1580] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:53:52 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:53.005797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424a901-c2f0-40e2-ab8d-134482fd3440 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.006184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424a901-c2f0-40e2-ab8d-134482fd3440 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.006570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424a901-c2f0-40e2-ab8d-134482fd3440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.023634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-5424a901-c2f0-40e2-ab8d-134482fd3440 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:53.024864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5424a901-c2f0-40e2-ab8d-134482fd3440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:53.039032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33b2b9-ddf1-4afe-9901-3d9c4b58c9c9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.043879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae09cf-4cc2-4d0d-b0ff-711700648b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.044562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae09cf-4cc2-4d0d-b0ff-711700648b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.044846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae09cf-4cc2-4d0d-b0ff-711700648b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.045123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae09cf-4cc2-4d0d-b0ff-711700648b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.045860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae09cf-4cc2-4d0d-b0ff-711700648b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.047625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae09cf-4cc2-4d0d-b0ff-711700648b96 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.055831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:53.057896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 784/1581] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.058690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae09cf-4cc2-4d0d-b0ff-711700648b96 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.063763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:53.068892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae09cf-4cc2-4d0d-b0ff-711700648b96 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.075398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65705424-ea41-418b-bac8-a1f937110120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.076294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705424-ea41-418b-bac8-a1f937110120 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.076654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705424-ea41-418b-bac8-a1f937110120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.077003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705424-ea41-418b-bac8-a1f937110120 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.077769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65705424-ea41-418b-bac8-a1f937110120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.078892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: WARNING py.warnings [None req-65705424-ea41-418b-bac8-a1f937110120 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:53:53.078892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: warnings.warn(msg) Jul 04 05:53:53.078892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: Jul 04 05:53:53.080170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65705424-ea41-418b-bac8-a1f937110120 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.089715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 798/1582] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:53.093203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424a901-c2f0-40e2-ab8d-134482fd3440 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.100421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424a901-c2f0-40e2-ab8d-134482fd3440 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.105875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce815762-55ff-4ff1-bf62-22796352e130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.106642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce815762-55ff-4ff1-bf62-22796352e130 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.109957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 785/1583] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:53:53 2022] POST /identity/v3/users => generated 292 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:53.110092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce815762-55ff-4ff1-bf62-22796352e130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.110092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce815762-55ff-4ff1-bf62-22796352e130 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.110299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65705424-ea41-418b-bac8-a1f937110120 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.110396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce815762-55ff-4ff1-bf62-22796352e130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.110515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce815762-55ff-4ff1-bf62-22796352e130 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.117911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65705424-ea41-418b-bac8-a1f937110120 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.129201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0787d92-6302-4af7-b8ad-7c54fc63cc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.130190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0787d92-6302-4af7-b8ad-7c54fc63cc1f admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.130351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0787d92-6302-4af7-b8ad-7c54fc63cc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.130620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0787d92-6302-4af7-b8ad-7c54fc63cc1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.131379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0787d92-6302-4af7-b8ad-7c54fc63cc1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.132974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0787d92-6302-4af7-b8ad-7c54fc63cc1f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.136319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 799/1584] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:53:53 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:53.136319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce815762-55ff-4ff1-bf62-22796352e130 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.143777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce815762-55ff-4ff1-bf62-22796352e130 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.146390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 786/1585] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.147177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0787d92-6302-4af7-b8ad-7c54fc63cc1f admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.151403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8c5a4-9723-42e9-ba49-3161d18f49db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.152069 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8c5a4-9723-42e9-ba49-3161d18f49db admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.152346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8c5a4-9723-42e9-ba49-3161d18f49db admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.152614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8c5a4-9723-42e9-ba49-3161d18f49db admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.153336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8c5a4-9723-42e9-ba49-3161d18f49db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.154957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8c5a4-9723-42e9-ba49-3161d18f49db admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.158906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0787d92-6302-4af7-b8ad-7c54fc63cc1f admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.164491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1affeab-e6a7-44e5-89ce-55bbba5fbf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.165165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1affeab-e6a7-44e5-89ce-55bbba5fbf8b admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.165443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1affeab-e6a7-44e5-89ce-55bbba5fbf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.165714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1affeab-e6a7-44e5-89ce-55bbba5fbf8b admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.165985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 800/1586] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.166971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8c5a4-9723-42e9-ba49-3161d18f49db admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.167894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1affeab-e6a7-44e5-89ce-55bbba5fbf8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.169939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1affeab-e6a7-44e5-89ce-55bbba5fbf8b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.177582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8c5a4-9723-42e9-ba49-3161d18f49db admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.182916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ed9f9-16ad-4af4-9887-05311d1d13eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.183562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ed9f9-16ad-4af4-9887-05311d1d13eb admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.183842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ed9f9-16ad-4af4-9887-05311d1d13eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.184106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ed9f9-16ad-4af4-9887-05311d1d13eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.184834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ed9f9-16ad-4af4-9887-05311d1d13eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.186452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ed9f9-16ad-4af4-9887-05311d1d13eb admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.195287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 787/1587] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:53:53 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:53.196135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1affeab-e6a7-44e5-89ce-55bbba5fbf8b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.197923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 801/1588] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.198850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ed9f9-16ad-4af4-9887-05311d1d13eb admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.206175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1affeab-e6a7-44e5-89ce-55bbba5fbf8b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.210792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6dd90-c23a-4246-9f44-27303d45b335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.211719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6dd90-c23a-4246-9f44-27303d45b335 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.211997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6dd90-c23a-4246-9f44-27303d45b335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.212269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6dd90-c23a-4246-9f44-27303d45b335 admin admin] PATH_INFO: `/v3/projects/3a72d2feb4844d9cba103aa34d7352ab/users/da26676819d4404fac31efeecf89d913/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.226783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6dd90-c23a-4246-9f44-27303d45b335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a72d2feb4844d9cba103aa34d7352ab, user_id=da26676819d4404fac31efeecf89d913, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.229491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6dd90-c23a-4246-9f44-27303d45b335 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.245466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 788/1589] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:53:53 2022] PUT /identity/v3/projects/3a72d2feb4844d9cba103aa34d7352ab/users/da26676819d4404fac31efeecf89d913/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:53.246369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6dd90-c23a-4246-9f44-27303d45b335 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.267071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ed9f9-16ad-4af4-9887-05311d1d13eb admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.307068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6dd90-c23a-4246-9f44-27303d45b335 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.312111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9b13f-d2d0-4ac9-a062-3dbcb9721f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.312837 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9b13f-d2d0-4ac9-a062-3dbcb9721f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.313127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9b13f-d2d0-4ac9-a062-3dbcb9721f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.313396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9b13f-d2d0-4ac9-a062-3dbcb9721f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.314119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9b13f-d2d0-4ac9-a062-3dbcb9721f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.315764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9b13f-d2d0-4ac9-a062-3dbcb9721f66 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.316711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be09fe-224a-49a5-aa66-d983f5e0ce53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.317361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be09fe-224a-49a5-aa66-d983f5e0ce53 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.317636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be09fe-224a-49a5-aa66-d983f5e0ce53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.317918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be09fe-224a-49a5-aa66-d983f5e0ce53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.325801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 789/1590] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.326599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9b13f-d2d0-4ac9-a062-3dbcb9721f66 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.327052 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be09fe-224a-49a5-aa66-d983f5e0ce53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.328643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be09fe-224a-49a5-aa66-d983f5e0ce53 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.335782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9b13f-d2d0-4ac9-a062-3dbcb9721f66 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.338638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 802/1591] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:53.339369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be09fe-224a-49a5-aa66-d983f5e0ce53 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.340359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b97183-fdc3-4999-af40-f98765b8d283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.341015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b97183-fdc3-4999-af40-f98765b8d283 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.341295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b97183-fdc3-4999-af40-f98765b8d283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.341569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b97183-fdc3-4999-af40-f98765b8d283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.342288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b97183-fdc3-4999-af40-f98765b8d283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.344049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b97183-fdc3-4999-af40-f98765b8d283 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.350151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be09fe-224a-49a5-aa66-d983f5e0ce53 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.357138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 790/1592] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.357138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b97183-fdc3-4999-af40-f98765b8d283 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.358060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb4b17-8ee0-46e2-87a7-a7947e30bb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.358804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb4b17-8ee0-46e2-87a7-a7947e30bb79 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.359184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb4b17-8ee0-46e2-87a7-a7947e30bb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.359542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb4b17-8ee0-46e2-87a7-a7947e30bb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.360336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb4b17-8ee0-46e2-87a7-a7947e30bb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.362062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb4b17-8ee0-46e2-87a7-a7947e30bb79 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.369431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b97183-fdc3-4999-af40-f98765b8d283 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.376463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 803/1593] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.376463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb4b17-8ee0-46e2-87a7-a7947e30bb79 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.377464 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1197d68-075f-441b-8353-cd4aedb93a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.378213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1197d68-075f-441b-8353-cd4aedb93a1d admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.378563 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1197d68-075f-441b-8353-cd4aedb93a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.378880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1197d68-075f-441b-8353-cd4aedb93a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.379738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1197d68-075f-441b-8353-cd4aedb93a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.381636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1197d68-075f-441b-8353-cd4aedb93a1d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.386841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb4b17-8ee0-46e2-87a7-a7947e30bb79 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.392484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e3c21-5945-4f14-bd50-279db222a986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.393277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e3c21-5945-4f14-bd50-279db222a986 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.393597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e3c21-5945-4f14-bd50-279db222a986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.394194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e3c21-5945-4f14-bd50-279db222a986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.394720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e3c21-5945-4f14-bd50-279db222a986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.396471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e3c21-5945-4f14-bd50-279db222a986 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.398389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 791/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.399524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1197d68-075f-441b-8353-cd4aedb93a1d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.407509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 804/1595] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.410551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b6b52-e9cb-4fc6-b1e9-127e49a9397d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.410551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b6b52-e9cb-4fc6-b1e9-127e49a9397d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.410551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b6b52-e9cb-4fc6-b1e9-127e49a9397d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.411657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1197d68-075f-441b-8353-cd4aedb93a1d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.416779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407eb0e-e7c4-4934-b08a-c86b4a1da50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.417483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407eb0e-e7c4-4934-b08a-c86b4a1da50d admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.417760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407eb0e-e7c4-4934-b08a-c86b4a1da50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.418001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407eb0e-e7c4-4934-b08a-c86b4a1da50d admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.418744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407eb0e-e7c4-4934-b08a-c86b4a1da50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.420391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407eb0e-e7c4-4934-b08a-c86b4a1da50d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.434903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 792/1596] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.434903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407eb0e-e7c4-4934-b08a-c86b4a1da50d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.444520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407eb0e-e7c4-4934-b08a-c86b4a1da50d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.445695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-741b6b52-e9cb-4fc6-b1e9-127e49a9397d None None] MFA Rules not processed for user `da26676819d4404fac31efeecf89d913`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:53.446997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-741b6b52-e9cb-4fc6-b1e9-127e49a9397d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:53.450722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8f362-9ca1-4e82-8a92-038a361efa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.451557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f362-9ca1-4e82-8a92-038a361efa2f admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.451949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f362-9ca1-4e82-8a92-038a361efa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.452611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f362-9ca1-4e82-8a92-038a361efa2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.453120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8f362-9ca1-4e82-8a92-038a361efa2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.454807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8f362-9ca1-4e82-8a92-038a361efa2f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.465594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 793/1597] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.468585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:53.469800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8f362-9ca1-4e82-8a92-038a361efa2f admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.480955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8f362-9ca1-4e82-8a92-038a361efa2f admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.486585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9160b08-4f47-44ab-bcc2-45df8fef9ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.487709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9160b08-4f47-44ab-bcc2-45df8fef9ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.487709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9160b08-4f47-44ab-bcc2-45df8fef9ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.487934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9160b08-4f47-44ab-bcc2-45df8fef9ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.488564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9160b08-4f47-44ab-bcc2-45df8fef9ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.490228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9160b08-4f47-44ab-bcc2-45df8fef9ea1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.509238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 794/1598] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.510342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9160b08-4f47-44ab-bcc2-45df8fef9ea1 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.526533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9160b08-4f47-44ab-bcc2-45df8fef9ea1 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.534349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 805/1599] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:53 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:53.534349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b6b52-e9cb-4fc6-b1e9-127e49a9397d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.535376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84501b0c-804c-45c6-944a-4682f71fc700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.536387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84501b0c-804c-45c6-944a-4682f71fc700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.536740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84501b0c-804c-45c6-944a-4682f71fc700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.537084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84501b0c-804c-45c6-944a-4682f71fc700 admin admin] PATH_INFO: `/v3/projects/cbb322e1350d44e496c53ffc9b869d7a/users/72cb1ae8aa5243fca51dcff137181004/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.546009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b6b52-e9cb-4fc6-b1e9-127e49a9397d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.551956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6dcb7-5354-4299-98bb-bb739f28655b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.553047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6dcb7-5354-4299-98bb-bb739f28655b admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.553328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6dcb7-5354-4299-98bb-bb739f28655b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.553684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6dcb7-5354-4299-98bb-bb739f28655b admin admin] PATH_INFO: `/v3/projects/f3251b70a6c64287acc95cc99d4e3b4d/users/982651a870c44a07bc4df322919e605e/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.555273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84501b0c-804c-45c6-944a-4682f71fc700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb322e1350d44e496c53ffc9b869d7a, user_id=72cb1ae8aa5243fca51dcff137181004, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.558668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84501b0c-804c-45c6-944a-4682f71fc700 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.575066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6dcb7-5354-4299-98bb-bb739f28655b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3251b70a6c64287acc95cc99d4e3b4d, user_id=982651a870c44a07bc4df322919e605e, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.578780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6dcb7-5354-4299-98bb-bb739f28655b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.583293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 795/1600] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:53:53 2022] PUT /identity/v3/projects/cbb322e1350d44e496c53ffc9b869d7a/users/72cb1ae8aa5243fca51dcff137181004/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:53.583293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84501b0c-804c-45c6-944a-4682f71fc700 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.604595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:53.607319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 806/1601] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:53:53 2022] PUT /identity/v3/projects/f3251b70a6c64287acc95cc99d4e3b4d/users/982651a870c44a07bc4df322919e605e/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:53.608156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6dcb7-5354-4299-98bb-bb739f28655b admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.669917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84501b0c-804c-45c6-944a-4682f71fc700 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.675482 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dae980-e457-4921-9967-925f44c3943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.676825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dae980-e457-4921-9967-925f44c3943e admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.676981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6dcb7-5354-4299-98bb-bb739f28655b admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.677818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dae980-e457-4921-9967-925f44c3943e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.678258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dae980-e457-4921-9967-925f44c3943e admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.679545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dae980-e457-4921-9967-925f44c3943e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.681560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dae980-e457-4921-9967-925f44c3943e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.687012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dc924-d25b-4eb4-8083-f82352d15796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.688007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc924-d25b-4eb4-8083-f82352d15796 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.688289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc924-d25b-4eb4-8083-f82352d15796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.688566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc924-d25b-4eb4-8083-f82352d15796 admin admin] PATH_INFO: `/v3/projects/a3b48b3027ae4e4e9428bc00d9617f8f/users/80c7837783194f5b8db43da205c80f2c/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.698691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 796/1602] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:53.700135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dae980-e457-4921-9967-925f44c3943e admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.704299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dc924-d25b-4eb4-8083-f82352d15796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b48b3027ae4e4e9428bc00d9617f8f, user_id=80c7837783194f5b8db43da205c80f2c, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.707003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dc924-d25b-4eb4-8083-f82352d15796 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.714581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dae980-e457-4921-9967-925f44c3943e admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.720370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a37671-5fdd-42f8-b1b1-cc159a358448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.721087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a37671-5fdd-42f8-b1b1-cc159a358448 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.721374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a37671-5fdd-42f8-b1b1-cc159a358448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.721735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a37671-5fdd-42f8-b1b1-cc159a358448 admin admin] PATH_INFO: `/v3/projects/4e16375231df4b768b8355e320e91731/users/e367c76dfa7f4d2791e60ce148f352c3/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.725625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 807/1603] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:53:53 2022] PUT /identity/v3/projects/a3b48b3027ae4e4e9428bc00d9617f8f/users/80c7837783194f5b8db43da205c80f2c/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:53.727385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dc924-d25b-4eb4-8083-f82352d15796 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.739652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a37671-5fdd-42f8-b1b1-cc159a358448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e16375231df4b768b8355e320e91731, user_id=e367c76dfa7f4d2791e60ce148f352c3, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.742406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a37671-5fdd-42f8-b1b1-cc159a358448 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.759723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 797/1604] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:53:53 2022] PUT /identity/v3/projects/4e16375231df4b768b8355e320e91731/users/e367c76dfa7f4d2791e60ce148f352c3/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:53.760706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a37671-5fdd-42f8-b1b1-cc159a358448 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:53.817193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dc924-d25b-4eb4-8083-f82352d15796 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.832567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a37671-5fdd-42f8-b1b1-cc159a358448 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:53.837171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ba917-3b2d-43ed-b705-048e2e0829e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.837850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ba917-3b2d-43ed-b705-048e2e0829e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.838130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ba917-3b2d-43ed-b705-048e2e0829e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.838404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ba917-3b2d-43ed-b705-048e2e0829e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.839315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ba917-3b2d-43ed-b705-048e2e0829e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.841667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ba917-3b2d-43ed-b705-048e2e0829e6 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.849183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:53.871586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 798/1605] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:53:53 2022] POST /identity/v3/projects => generated 355 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:53.876076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94690321-e092-4726-aae9-c642195f95a6 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.876463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94690321-e092-4726-aae9-c642195f95a6 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.876879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94690321-e092-4726-aae9-c642195f95a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.885046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44678a-8a32-4776-8275-3eb9c2dd0b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:53.885942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44678a-8a32-4776-8275-3eb9c2dd0b0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.886217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44678a-8a32-4776-8275-3eb9c2dd0b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.886486 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44678a-8a32-4776-8275-3eb9c2dd0b0e admin admin] PATH_INFO: `/v3/projects/b63ffe62990842e4b20b86218bf26dc3/users/c6febb3bb852434fb29d0ef3c59cfa7c/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.901764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44678a-8a32-4776-8275-3eb9c2dd0b0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63ffe62990842e4b20b86218bf26dc3, user_id=c6febb3bb852434fb29d0ef3c59cfa7c, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:53.904731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44678a-8a32-4776-8275-3eb9c2dd0b0e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:53.905214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-94690321-e092-4726-aae9-c642195f95a6 None None] MFA Rules not processed for user `72cb1ae8aa5243fca51dcff137181004`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:53.906732 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-94690321-e092-4726-aae9-c642195f95a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:53.935070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 808/1606] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:53:53 2022] PUT /identity/v3/projects/b63ffe62990842e4b20b86218bf26dc3/users/c6febb3bb852434fb29d0ef3c59cfa7c/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:53.935070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90d5bc-53dd-4890-ab72-4088bb2737e9 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:53.935070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90d5bc-53dd-4890-ab72-4088bb2737e9 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:53.935070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90d5bc-53dd-4890-ab72-4088bb2737e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:53.950778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ac90d5bc-53dd-4890-ab72-4088bb2737e9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:53.952000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ac90d5bc-53dd-4890-ab72-4088bb2737e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:53.984364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:54.003704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 799/1607] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:54.004862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94690321-e092-4726-aae9-c642195f95a6 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:54.028946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 809/1608] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:54.030685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e2403-8c6b-402e-b209-658191c25855 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.031034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e2403-8c6b-402e-b209-658191c25855 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.032132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e2403-8c6b-402e-b209-658191c25855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.077403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d39e2403-8c6b-402e-b209-658191c25855 None None] MFA Rules not processed for user `80c7837783194f5b8db43da205c80f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:54.079319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d39e2403-8c6b-402e-b209-658191c25855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:54.110363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94690321-e092-4726-aae9-c642195f95a6 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:54.122264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dbe11-b3fb-4ef9-94ca-4e39dad3f1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:54.126780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dbe11-b3fb-4ef9-94ca-4e39dad3f1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.127345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dbe11-b3fb-4ef9-94ca-4e39dad3f1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.128262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dbe11-b3fb-4ef9-94ca-4e39dad3f1a4 admin admin] PATH_INFO: `/v3/projects/bdd96113d4ef451d9ffc81c03c6ea70e/users/32703b5451cf48b09966a7412b747099/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.148399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267dbe11-b3fb-4ef9-94ca-4e39dad3f1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdd96113d4ef451d9ffc81c03c6ea70e, user_id=32703b5451cf48b09966a7412b747099, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:54.151633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267dbe11-b3fb-4ef9-94ca-4e39dad3f1a4 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:54.174756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 800/1609] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:53:54 2022] PUT /identity/v3/projects/bdd96113d4ef451d9ffc81c03c6ea70e/users/32703b5451cf48b09966a7412b747099/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:54.176043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 810/1610] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:54.177071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c51d9-988f-4926-b04f-1a31f2ce8e39 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.177280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e2403-8c6b-402e-b209-658191c25855 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:54.177392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c51d9-988f-4926-b04f-1a31f2ce8e39 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.177622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c51d9-988f-4926-b04f-1a31f2ce8e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.195238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e2403-8c6b-402e-b209-658191c25855 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:54.197653 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-3a7c51d9-988f-4926-b04f-1a31f2ce8e39 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:54.198870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3a7c51d9-988f-4926-b04f-1a31f2ce8e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:54.257978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:54.283171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed2019-2ef7-4bb5-aa02-52a092032004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:54.284387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed2019-2ef7-4bb5-aa02-52a092032004 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.284880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed2019-2ef7-4bb5-aa02-52a092032004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.285384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed2019-2ef7-4bb5-aa02-52a092032004 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.286494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 801/1611] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:54.286788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed2019-2ef7-4bb5-aa02-52a092032004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:54.287955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6db73e-bc99-4049-8c8b-b372488de51c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.288248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6db73e-bc99-4049-8c8b-b372488de51c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.288695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6db73e-bc99-4049-8c8b-b372488de51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.290265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed2019-2ef7-4bb5-aa02-52a092032004 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:54.320332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 811/1612] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:53:54 2022] POST /identity/v3/users => generated 284 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:53:54.321185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2b6db73e-bc99-4049-8c8b-b372488de51c None None] MFA Rules not processed for user `c6febb3bb852434fb29d0ef3c59cfa7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:54.322667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2b6db73e-bc99-4049-8c8b-b372488de51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:54.323747 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf443db5-aed2-4b94-bac9-d284474400c5 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.324267 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf443db5-aed2-4b94-bac9-d284474400c5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.324797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf443db5-aed2-4b94-bac9-d284474400c5 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.332339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 812/1613] 10.222.0.42 () {54 vars in 961 bytes} [Mon Jul 4 05:53:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:53:54.333252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf443db5-aed2-4b94-bac9-d284474400c5 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:54.407025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 802/1614] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:54.408711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4c521-e277-4b8b-9448-281da362a98b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.409005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4c521-e277-4b8b-9448-281da362a98b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.409280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4c521-e277-4b8b-9448-281da362a98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.416788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf443db5-aed2-4b94-bac9-d284474400c5 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:54.421422 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505f427-b7ff-4033-bb72-2c65a2b52629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:54.422416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f427-b7ff-4033-bb72-2c65a2b52629 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.422703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f427-b7ff-4033-bb72-2c65a2b52629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.422974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f427-b7ff-4033-bb72-2c65a2b52629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.430331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-1ee4c521-e277-4b8b-9448-281da362a98b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:54.432074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1ee4c521-e277-4b8b-9448-281da362a98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:54.434638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505f427-b7ff-4033-bb72-2c65a2b52629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:54.436419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505f427-b7ff-4033-bb72-2c65a2b52629 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:54.444591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 803/1615] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:54.444591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4a384-1304-4c83-8681-7717bd6029c7 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.444591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4a384-1304-4c83-8681-7717bd6029c7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.444591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4a384-1304-4c83-8681-7717bd6029c7 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.447200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 804/1616] 10.222.0.42 () {54 vars in 961 bytes} [Mon Jul 4 05:53:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:53:54.448014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4a384-1304-4c83-8681-7717bd6029c7 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:54.453788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:54.456504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 813/1617] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:54.457870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505f427-b7ff-4033-bb72-2c65a2b52629 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:54.460446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4a384-1304-4c83-8681-7717bd6029c7 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:54.465390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833daaf-a518-4011-89dc-041f5c39849d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:54.466420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833daaf-a518-4011-89dc-041f5c39849d service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.466808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833daaf-a518-4011-89dc-041f5c39849d service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.467173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833daaf-a518-4011-89dc-041f5c39849d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.469380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505f427-b7ff-4033-bb72-2c65a2b52629 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:54.474892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fe4ee-7d3e-4c45-a601-e0235533076f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:54.475606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fe4ee-7d3e-4c45-a601-e0235533076f admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.476055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fe4ee-7d3e-4c45-a601-e0235533076f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.476335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fe4ee-7d3e-4c45-a601-e0235533076f admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.477063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fe4ee-7d3e-4c45-a601-e0235533076f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:54.479671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fe4ee-7d3e-4c45-a601-e0235533076f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:54.480732 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833daaf-a518-4011-89dc-041f5c39849d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:54.482665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833daaf-a518-4011-89dc-041f5c39849d service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:54.492222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 814/1618] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:54.493726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ff58e-555a-4356-ab19-b10b2db46f2c None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.494200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ff58e-555a-4356-ab19-b10b2db46f2c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.494477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ff58e-555a-4356-ab19-b10b2db46f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.497778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 805/1619] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:54.503486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833daaf-a518-4011-89dc-041f5c39849d service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:54.516558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833daaf-a518-4011-89dc-041f5c39849d service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:54.520752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c59e2-3a41-4a8d-9231-71cfc8d47e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:54.522217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c59e2-3a41-4a8d-9231-71cfc8d47e38 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.522546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c59e2-3a41-4a8d-9231-71cfc8d47e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.522851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c59e2-3a41-4a8d-9231-71cfc8d47e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.534120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c59e2-3a41-4a8d-9231-71cfc8d47e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:54.536123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c59e2-3a41-4a8d-9231-71cfc8d47e38 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:54.548108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 806/1620] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:54.549342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-917ff58e-555a-4356-ab19-b10b2db46f2c None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:54.550071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba388344-3e5c-412a-a167-6f275dfd1b03 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.550382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba388344-3e5c-412a-a167-6f275dfd1b03 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.550682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba388344-3e5c-412a-a167-6f275dfd1b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.550793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-917ff58e-555a-4356-ab19-b10b2db46f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:54.588436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ba388344-3e5c-412a-a167-6f275dfd1b03 None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:54.590145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ba388344-3e5c-412a-a167-6f275dfd1b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:54.673636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 815/1621] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Jul 4 05:53:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:54.674660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ff58e-555a-4356-ab19-b10b2db46f2c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:54.691451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ff58e-555a-4356-ab19-b10b2db46f2c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:54.700122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b0e77-9554-4d0e-8f00-5bf5c2c39642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:54.701282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b0e77-9554-4d0e-8f00-5bf5c2c39642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.701668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b0e77-9554-4d0e-8f00-5bf5c2c39642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.702036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b0e77-9554-4d0e-8f00-5bf5c2c39642 admin admin] PATH_INFO: `/v3/projects/481ba4ca7a10481d988c7434a321c2e0/users/6ed165c990d24da2983e5d08960f841b/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.725284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b0e77-9554-4d0e-8f00-5bf5c2c39642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=481ba4ca7a10481d988c7434a321c2e0, user_id=6ed165c990d24da2983e5d08960f841b, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:54.737966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b0e77-9554-4d0e-8f00-5bf5c2c39642 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:54.763105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 816/1622] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:53:54 2022] PUT /identity/v3/projects/481ba4ca7a10481d988c7434a321c2e0/users/6ed165c990d24da2983e5d08960f841b/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:54.766466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 807/1623] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Jul 4 05:53:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:54.767000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b0e77-9554-4d0e-8f00-5bf5c2c39642 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:54.768376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba388344-3e5c-412a-a167-6f275dfd1b03 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:54.782533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba388344-3e5c-412a-a167-6f275dfd1b03 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:54.863104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96516a-bad8-45dc-8374-28712121733d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:54.863880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96516a-bad8-45dc-8374-28712121733d admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.864166 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96516a-bad8-45dc-8374-28712121733d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.864442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96516a-bad8-45dc-8374-28712121733d admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:54.865698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96516a-bad8-45dc-8374-28712121733d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:54.867507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96516a-bad8-45dc-8374-28712121733d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:54.867655 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:54.881023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 808/1624] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:53:54.882711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96516a-bad8-45dc-8374-28712121733d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:54.969996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b0e77-9554-4d0e-8f00-5bf5c2c39642 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:54.985291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455a647-b6b9-45f6-a47b-7630e2ce5308 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:54.986120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455a647-b6b9-45f6-a47b-7630e2ce5308 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:54.986410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455a647-b6b9-45f6-a47b-7630e2ce5308 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:54.986689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455a647-b6b9-45f6-a47b-7630e2ce5308 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:55.025675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:55.035829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96516a-bad8-45dc-8374-28712121733d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:55.043228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5968e6-6a9c-4aba-8fa6-be0ae1ed8624 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:55.044429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5968e6-6a9c-4aba-8fa6-be0ae1ed8624 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:55.045128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5968e6-6a9c-4aba-8fa6-be0ae1ed8624 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:55.045520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5968e6-6a9c-4aba-8fa6-be0ae1ed8624 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:55.170478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455a647-b6b9-45f6-a47b-7630e2ce5308 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:55.173613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455a647-b6b9-45f6-a47b-7630e2ce5308 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:55.190835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 817/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:55.191798 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455a647-b6b9-45f6-a47b-7630e2ce5308 service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:55.208631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455a647-b6b9-45f6-a47b-7630e2ce5308 service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:55.220129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cfd6f-2251-4c81-9b27-f6b961e173b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:55.221570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cfd6f-2251-4c81-9b27-f6b961e173b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:55.221854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cfd6f-2251-4c81-9b27-f6b961e173b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:55.222183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cfd6f-2251-4c81-9b27-f6b961e173b1 admin admin] PATH_INFO: `/v3/projects/481ba4ca7a10481d988c7434a321c2e0/users/6ed165c990d24da2983e5d08960f841b/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:55.229257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cfd6f-2251-4c81-9b27-f6b961e173b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=481ba4ca7a10481d988c7434a321c2e0, user_id=6ed165c990d24da2983e5d08960f841b, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:55.229442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5968e6-6a9c-4aba-8fa6-be0ae1ed8624 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:55.230962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5968e6-6a9c-4aba-8fa6-be0ae1ed8624 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:55.243112 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cfd6f-2251-4c81-9b27-f6b961e173b1 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:55.258997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 809/1626] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:55.270856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 818/1627] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:53:55 2022] PUT /identity/v3/projects/481ba4ca7a10481d988c7434a321c2e0/users/6ed165c990d24da2983e5d08960f841b/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:55.282216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dd36c-5a85-4488-a0d4-5097680380c1 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:55.282526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dd36c-5a85-4488-a0d4-5097680380c1 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:55.282778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dd36c-5a85-4488-a0d4-5097680380c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:55.315296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-844dd36c-5a85-4488-a0d4-5097680380c1 None None] MFA Rules not processed for user `6ed165c990d24da2983e5d08960f841b`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:55.316950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-844dd36c-5a85-4488-a0d4-5097680380c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:55.470215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 810/1628] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:55 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:55.484347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cfd6f-2251-4c81-9b27-f6b961e173b1 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:55.602937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cfd6f-2251-4c81-9b27-f6b961e173b1 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:55.612071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a77b0-e385-4d41-94b8-c8a2201e2c0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:55.613068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a77b0-e385-4d41-94b8-c8a2201e2c0a service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:55.613068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a77b0-e385-4d41-94b8-c8a2201e2c0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:55.613305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a77b0-e385-4d41-94b8-c8a2201e2c0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:55.706626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a77b0-e385-4d41-94b8-c8a2201e2c0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:55.706626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a77b0-e385-4d41-94b8-c8a2201e2c0a service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:55.707882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7f30a-0e10-49cc-8db4-9a13db74f462 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:55.707882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7f30a-0e10-49cc-8db4-9a13db74f462 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:55.707882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7f30a-0e10-49cc-8db4-9a13db74f462 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:55.707882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 811/1629] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jul 4 05:53:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:53:55.708297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 819/1630] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:55 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:55.708387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da3aa4-19ca-444e-af0f-0e0e48000772 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:55.708387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da3aa4-19ca-444e-af0f-0e0e48000772 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:55.708387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da3aa4-19ca-444e-af0f-0e0e48000772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:55.759807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c3da3aa4-19ca-444e-af0f-0e0e48000772 None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:55.761053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c3da3aa4-19ca-444e-af0f-0e0e48000772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:55.808208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a77b0-e385-4d41-94b8-c8a2201e2c0a service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:55.897665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 812/1631] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 05:53:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:55.898423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da3aa4-19ca-444e-af0f-0e0e48000772 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:55.900274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:55.914299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da3aa4-19ca-444e-af0f-0e0e48000772 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:55.920205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f164e79-a8f7-4c9d-8b76-2be19c7cbe67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:55.921258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f164e79-a8f7-4c9d-8b76-2be19c7cbe67 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:55.921607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f164e79-a8f7-4c9d-8b76-2be19c7cbe67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:55.921879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f164e79-a8f7-4c9d-8b76-2be19c7cbe67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:55.923371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a77b0-e385-4d41-94b8-c8a2201e2c0a service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:55.930034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e3732-1ec3-4078-9cdd-b87dc868118e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:55.930771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e3732-1ec3-4078-9cdd-b87dc868118e service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:55.931055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e3732-1ec3-4078-9cdd-b87dc868118e service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:55.931368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e3732-1ec3-4078-9cdd-b87dc868118e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:55.937186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:55.987664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f164e79-a8f7-4c9d-8b76-2be19c7cbe67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:55.989408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f164e79-a8f7-4c9d-8b76-2be19c7cbe67 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:55.993784 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e3732-1ec3-4078-9cdd-b87dc868118e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:55.995410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e3732-1ec3-4078-9cdd-b87dc868118e service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:55.999306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 813/1632] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:55 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:56.000187 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f164e79-a8f7-4c9d-8b76-2be19c7cbe67 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:56.006298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 820/1633] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:56.008688 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f164e79-a8f7-4c9d-8b76-2be19c7cbe67 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:56.012644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51232b67-b265-44ae-a5b3-4e3f760b9f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:56.013278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51232b67-b265-44ae-a5b3-4e3f760b9f3a service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:56.013554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51232b67-b265-44ae-a5b3-4e3f760b9f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:56.013824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51232b67-b265-44ae-a5b3-4e3f760b9f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:56.027508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51232b67-b265-44ae-a5b3-4e3f760b9f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:56.029587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51232b67-b265-44ae-a5b3-4e3f760b9f3a service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:56.043353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 814/1634] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:56.060524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ed30d-01e6-40a2-a90f-06d8998979a2 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:56.060841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ed30d-01e6-40a2-a90f-06d8998979a2 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:56.061106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ed30d-01e6-40a2-a90f-06d8998979a2 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:56.063106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 821/1635] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jul 4 05:53:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:53:56.071958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f5435-600d-4761-b672-7b31920be5f5 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:56.073198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f5435-600d-4761-b672-7b31920be5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:56.073774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f5435-600d-4761-b672-7b31920be5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:56.095434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-fb8f5435-600d-4761-b672-7b31920be5f5 None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:56.096762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fb8f5435-600d-4761-b672-7b31920be5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:56.109186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 815/1636] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 05:53:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:56.300525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ed30d-01e6-40a2-a90f-06d8998979a2 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:56.324667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ed30d-01e6-40a2-a90f-06d8998979a2 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:56.328858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a8ae2-f72f-4639-a959-8bee4e2494c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:56.329530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a8ae2-f72f-4639-a959-8bee4e2494c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:56.329962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a8ae2-f72f-4639-a959-8bee4e2494c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:56.330249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a8ae2-f72f-4639-a959-8bee4e2494c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:56.346675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a8ae2-f72f-4639-a959-8bee4e2494c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:56.349873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a8ae2-f72f-4639-a959-8bee4e2494c1 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:56.372687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 822/1637] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:57.486635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f5435-600d-4761-b672-7b31920be5f5 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:57.599076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f5435-600d-4761-b672-7b31920be5f5 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:57.607194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:57.608390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:57.608742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:57.609422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] PATH_INFO: `/v3/users/80c7837783194f5b8db43da205c80f2c` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:57.611640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c7837783194f5b8db43da205c80f2c)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:57.615093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:57.717897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c7837783194f5b8db43da205c80f2c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:53:57.728904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c7837783194f5b8db43da205c80f2c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:53:57.742841 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c7837783194f5b8db43da205c80f2c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:53:57.744739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c7837783194f5b8db43da205c80f2c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:53:57.745630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c7837783194f5b8db43da205c80f2c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:53:57.781535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 816/1638] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:53:57 2022] DELETE /identity/v3/users/80c7837783194f5b8db43da205c80f2c => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:57.786599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc13c8-d235-47d7-941b-e7bb8091074c None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:57.787761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc13c8-d235-47d7-941b-e7bb8091074c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:57.788191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc13c8-d235-47d7-941b-e7bb8091074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:57.818836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-51dc13c8-d235-47d7-941b-e7bb8091074c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:57.822283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-51dc13c8-d235-47d7-941b-e7bb8091074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:57.959375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 823/1639] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:57.980837 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:57.982739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:58.038462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:53:58.172733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a2448-f0bb-4742-9b61-f5631b354a41 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:58.178667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd13890-97bc-48cc-999a-2c66b639bd3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:58.179849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd13890-97bc-48cc-999a-2c66b639bd3d service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:58.180352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd13890-97bc-48cc-999a-2c66b639bd3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:58.181071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd13890-97bc-48cc-999a-2c66b639bd3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:58.194307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd13890-97bc-48cc-999a-2c66b639bd3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:58.196478 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd13890-97bc-48cc-999a-2c66b639bd3d service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:58.218693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 817/1640] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:53:59.526207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc13c8-d235-47d7-941b-e7bb8091074c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:53:59.528528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-51dc13c8-d235-47d7-941b-e7bb8091074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:59.553999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc13c8-d235-47d7-941b-e7bb8091074c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:53:59.574706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:53:59.575652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:59.575934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:59.578201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] PATH_INFO: `/v3/projects/a3b48b3027ae4e4e9428bc00d9617f8f` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:59.584957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3b48b3027ae4e4e9428bc00d9617f8f)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:53:59.591334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:53:59.710024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3b48b3027ae4e4e9428bc00d9617f8f', 'request_id': 'req-078f52af-3384-4255-8251-2870322ab8a4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:53:59.726180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3b48b3027ae4e4e9428bc00d9617f8f', 'request_id': 'req-078f52af-3384-4255-8251-2870322ab8a4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:53:59.738936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3b48b3027ae4e4e9428bc00d9617f8f', 'request_id': 'req-078f52af-3384-4255-8251-2870322ab8a4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:53:59.773892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] The token cache is being invalidate because project a3b48b3027ae4e4e9428bc00d9617f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:53:59.774228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:53:59.775141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-078f52af-3384-4255-8251-2870322ab8a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:53:59.779519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 824/1641] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:53:59 2022] DELETE /identity/v3/projects/a3b48b3027ae4e4e9428bc00d9617f8f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:53:59.782677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256aa05-1b16-43af-89b3-3fa45d809d13 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:59.783212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256aa05-1b16-43af-89b3-3fa45d809d13 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:59.783212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256aa05-1b16-43af-89b3-3fa45d809d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:59.815679 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-7256aa05-1b16-43af-89b3-3fa45d809d13 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:53:59.816852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7256aa05-1b16-43af-89b3-3fa45d809d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:53:59.976285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 818/1642] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:53:59.977344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b68b3-b252-4e50-852f-254bac739f08 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:53:59.977637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b68b3-b252-4e50-852f-254bac739f08 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:53:59.977920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b68b3-b252-4e50-852f-254bac739f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:53:59.992571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:00.006318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-bc2b68b3-b252-4e50-852f-254bac739f08 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:00.008549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-bc2b68b3-b252-4e50-852f-254bac739f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:00.020030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 825/1643] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:00.024626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256aa05-1b16-43af-89b3-3fa45d809d13 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:00.036834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256aa05-1b16-43af-89b3-3fa45d809d13 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:00.042543 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cea45-6471-4929-9adc-469db0c36a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:00.043665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cea45-6471-4929-9adc-469db0c36a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:00.044262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cea45-6471-4929-9adc-469db0c36a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:00.044718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cea45-6471-4929-9adc-469db0c36a00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:00.046035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cea45-6471-4929-9adc-469db0c36a00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:00.046643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cea45-6471-4929-9adc-469db0c36a00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:00.048582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cea45-6471-4929-9adc-469db0c36a00 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:00.072165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b68b3-b252-4e50-852f-254bac739f08 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:00.072771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 819/1644] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:00.087587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b68b3-b252-4e50-852f-254bac739f08 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:00.101626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6b7f3-1e01-478b-b129-9dff4b67dc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:00.102322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6b7f3-1e01-478b-b129-9dff4b67dc92 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:00.102608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6b7f3-1e01-478b-b129-9dff4b67dc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:00.102884 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6b7f3-1e01-478b-b129-9dff4b67dc92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:00.103619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6b7f3-1e01-478b-b129-9dff4b67dc92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:00.105589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6b7f3-1e01-478b-b129-9dff4b67dc92 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:00.159267 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 826/1645] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:00 2022] POST /identity/v3/projects => generated 405 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:00.164141 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cea45-6471-4929-9adc-469db0c36a00 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:00.183317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cea45-6471-4929-9adc-469db0c36a00 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:00.321135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077799-ddd2-474b-8511-67cec7e9f15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:00.322264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077799-ddd2-474b-8511-67cec7e9f15f admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:00.322753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077799-ddd2-474b-8511-67cec7e9f15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:00.323335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077799-ddd2-474b-8511-67cec7e9f15f admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:00.324259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2077799-ddd2-474b-8511-67cec7e9f15f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:00.327175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2077799-ddd2-474b-8511-67cec7e9f15f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:00.358359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 820/1646] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:00 2022] POST /identity/v3/users => generated 303 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:00.362172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6b7f3-1e01-478b-b129-9dff4b67dc92 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:00.380560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6b7f3-1e01-478b-b129-9dff4b67dc92 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:00.397136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a43d4-f0ef-40a8-b445-b1c2218f7179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:00.398007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a43d4-f0ef-40a8-b445-b1c2218f7179 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:00.398497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a43d4-f0ef-40a8-b445-b1c2218f7179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:00.398917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a43d4-f0ef-40a8-b445-b1c2218f7179 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:00.399717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a43d4-f0ef-40a8-b445-b1c2218f7179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:00.404228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a43d4-f0ef-40a8-b445-b1c2218f7179 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:00.438126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 827/1647] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:00.441818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077799-ddd2-474b-8511-67cec7e9f15f admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:00.483970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077799-ddd2-474b-8511-67cec7e9f15f admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:00.502794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11c0fb-1b32-4650-b499-f3afc1f9fcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:00.503543 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11c0fb-1b32-4650-b499-f3afc1f9fcec admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:00.503827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11c0fb-1b32-4650-b499-f3afc1f9fcec admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:00.504105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11c0fb-1b32-4650-b499-f3afc1f9fcec admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:00.504852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11c0fb-1b32-4650-b499-f3afc1f9fcec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:00.506696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11c0fb-1b32-4650-b499-f3afc1f9fcec admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:00.539598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 821/1648] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:00.543585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a43d4-f0ef-40a8-b445-b1c2218f7179 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:00.554846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a43d4-f0ef-40a8-b445-b1c2218f7179 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:00.565731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa18b1-336d-4595-8fe3-e8747ceda7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:00.567053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa18b1-336d-4595-8fe3-e8747ceda7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:00.567521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa18b1-336d-4595-8fe3-e8747ceda7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:00.567943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa18b1-336d-4595-8fe3-e8747ceda7f0 admin admin] PATH_INFO: `/v3/projects/079c79c5dc674e4988dcfc371ba9901e/users/672f5c7b82d94040b128b42b54072266/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:00.589770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa18b1-336d-4595-8fe3-e8747ceda7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079c79c5dc674e4988dcfc371ba9901e, user_id=672f5c7b82d94040b128b42b54072266, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:00.593177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa18b1-336d-4595-8fe3-e8747ceda7f0 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:00.628021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 828/1649] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:00 2022] PUT /identity/v3/projects/079c79c5dc674e4988dcfc371ba9901e/users/672f5c7b82d94040b128b42b54072266/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:00.638422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce7a4e-42c5-496a-92a4-c5851dd3aa40 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:00.638723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce7a4e-42c5-496a-92a4-c5851dd3aa40 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:00.638981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce7a4e-42c5-496a-92a4-c5851dd3aa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:00.677927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-e4ce7a4e-42c5-496a-92a4-c5851dd3aa40 None None] MFA Rules not processed for user `672f5c7b82d94040b128b42b54072266`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:00.680412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e4ce7a4e-42c5-496a-92a4-c5851dd3aa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:00.704411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:00.861091 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 822/1650] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:00.866775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa18b1-336d-4595-8fe3-e8747ceda7f0 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:01.044777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa18b1-336d-4595-8fe3-e8747ceda7f0 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:01.081401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cc588-21b9-42a7-85fd-8764320d9b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:01.081906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cc588-21b9-42a7-85fd-8764320d9b14 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:01.082185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cc588-21b9-42a7-85fd-8764320d9b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:01.082461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cc588-21b9-42a7-85fd-8764320d9b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:01.083174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cc588-21b9-42a7-85fd-8764320d9b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:01.085194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cc588-21b9-42a7-85fd-8764320d9b14 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:01.161465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 829/1651] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:00 2022] POST /identity/v3/projects => generated 405 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:01.165552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce7a4e-42c5-496a-92a4-c5851dd3aa40 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:01.208027 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce7a4e-42c5-496a-92a4-c5851dd3aa40 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:01.467605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c9f37-7fff-480e-9c58-d1df847266fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:01.468383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c9f37-7fff-480e-9c58-d1df847266fc admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:01.470052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c9f37-7fff-480e-9c58-d1df847266fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:01.470421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c9f37-7fff-480e-9c58-d1df847266fc admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:01.475742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c9f37-7fff-480e-9c58-d1df847266fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:01.485869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c9f37-7fff-480e-9c58-d1df847266fc admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:01.548410 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 823/1652] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:01 2022] POST /identity/v3/users => generated 309 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:01.553015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cc588-21b9-42a7-85fd-8764320d9b14 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:01.568404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cc588-21b9-42a7-85fd-8764320d9b14 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:01.576455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89245a-5d9b-4e52-a5ce-b917f9c31972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:01.579824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89245a-5d9b-4e52-a5ce-b917f9c31972 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:01.580113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89245a-5d9b-4e52-a5ce-b917f9c31972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:01.580392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89245a-5d9b-4e52-a5ce-b917f9c31972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:01.581167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89245a-5d9b-4e52-a5ce-b917f9c31972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:01.588021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89245a-5d9b-4e52-a5ce-b917f9c31972 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:01.640116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c9f37-7fff-480e-9c58-d1df847266fc admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:01.647889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 830/1653] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:01.661077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c9f37-7fff-480e-9c58-d1df847266fc admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:01.667741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f91eb7-663b-4e2c-9604-34a1e4cef580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:01.669160 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f91eb7-663b-4e2c-9604-34a1e4cef580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:01.670752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f91eb7-663b-4e2c-9604-34a1e4cef580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:01.673375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f91eb7-663b-4e2c-9604-34a1e4cef580 admin admin] PATH_INFO: `/v3/projects/fdd60ea786604e6da727ea31ed373c93/users/960c516cd96d42b5ad531c8b7c74fa79/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:01.713893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f91eb7-663b-4e2c-9604-34a1e4cef580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd60ea786604e6da727ea31ed373c93, user_id=960c516cd96d42b5ad531c8b7c74fa79, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:01.722401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f91eb7-663b-4e2c-9604-34a1e4cef580 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:01.758928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 824/1654] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:01 2022] PUT /identity/v3/projects/fdd60ea786604e6da727ea31ed373c93/users/960c516cd96d42b5ad531c8b7c74fa79/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:01.764842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89245a-5d9b-4e52-a5ce-b917f9c31972 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:01.799050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89245a-5d9b-4e52-a5ce-b917f9c31972 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:01.996250 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a50c7-3780-4ed8-8e64-f3bbdc4edb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:01.997734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a50c7-3780-4ed8-8e64-f3bbdc4edb1a admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:01.998390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a50c7-3780-4ed8-8e64-f3bbdc4edb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:01.999508 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a50c7-3780-4ed8-8e64-f3bbdc4edb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:02.000803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a50c7-3780-4ed8-8e64-f3bbdc4edb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:02.007067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a50c7-3780-4ed8-8e64-f3bbdc4edb1a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:02.089755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 831/1655] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:02.094395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f91eb7-663b-4e2c-9604-34a1e4cef580 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:02.112620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f91eb7-663b-4e2c-9604-34a1e4cef580 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:02.127683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecdd14-1925-4f5f-ad3b-48bea26628b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:02.129770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecdd14-1925-4f5f-ad3b-48bea26628b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:02.130073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecdd14-1925-4f5f-ad3b-48bea26628b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:02.130347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecdd14-1925-4f5f-ad3b-48bea26628b1 admin admin] PATH_INFO: `/v3/projects/fdd60ea786604e6da727ea31ed373c93/users/960c516cd96d42b5ad531c8b7c74fa79/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:02.165033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecdd14-1925-4f5f-ad3b-48bea26628b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd60ea786604e6da727ea31ed373c93, user_id=960c516cd96d42b5ad531c8b7c74fa79, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:02.190530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecdd14-1925-4f5f-ad3b-48bea26628b1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:02.230230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 825/1656] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:02 2022] PUT /identity/v3/projects/fdd60ea786604e6da727ea31ed373c93/users/960c516cd96d42b5ad531c8b7c74fa79/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:02.243619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e51f1-c518-4ea3-a195-8ede2ffd5c22 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:02.243913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e51f1-c518-4ea3-a195-8ede2ffd5c22 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:02.244196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e51f1-c518-4ea3-a195-8ede2ffd5c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:02.292208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:02.315492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-691e51f1-c518-4ea3-a195-8ede2ffd5c22 None None] MFA Rules not processed for user `960c516cd96d42b5ad531c8b7c74fa79`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:02.316751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-691e51f1-c518-4ea3-a195-8ede2ffd5c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:02.618524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 832/1657] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:02 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:02.619130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecdd14-1925-4f5f-ad3b-48bea26628b1 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:02.621789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-12ecdd14-1925-4f5f-ad3b-48bea26628b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:02.879615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecdd14-1925-4f5f-ad3b-48bea26628b1 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:02.891378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933519c-f8ec-4947-8020-7dc638b62049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:02.892321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933519c-f8ec-4947-8020-7dc638b62049 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:02.894742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933519c-f8ec-4947-8020-7dc638b62049 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:02.895275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933519c-f8ec-4947-8020-7dc638b62049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:02.920047 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933519c-f8ec-4947-8020-7dc638b62049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:02.921759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933519c-f8ec-4947-8020-7dc638b62049 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:02.968957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 826/1658] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:02 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:03.251352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e51f1-c518-4ea3-a195-8ede2ffd5c22 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:03.354326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933519c-f8ec-4947-8020-7dc638b62049 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:03.356563 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7933519c-f8ec-4947-8020-7dc638b62049 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:03.379526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:03.417181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e51f1-c518-4ea3-a195-8ede2ffd5c22 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:03.439344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:03.440394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:03.440675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:03.440948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] PATH_INFO: `/v3/users/672f5c7b82d94040b128b42b54072266` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:03.447867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=672f5c7b82d94040b128b42b54072266)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:03.448943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: WARNING py.warnings [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:54:03.448943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: warnings.warn(msg) Jul 04 05:54:03.448943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: Jul 04 05:54:03.456257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:03.599910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933519c-f8ec-4947-8020-7dc638b62049 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:03.606219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:03.614450 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:03.615868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:03.616381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] PATH_INFO: `/v3/users/c6febb3bb852434fb29d0ef3c59cfa7c` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:03.618633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6febb3bb852434fb29d0ef3c59cfa7c)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:03.621025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:03.706206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '672f5c7b82d94040b128b42b54072266'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:03.748408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6febb3bb852434fb29d0ef3c59cfa7c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:03.759909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '672f5c7b82d94040b128b42b54072266'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:03.763813 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '672f5c7b82d94040b128b42b54072266'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:03.768999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6febb3bb852434fb29d0ef3c59cfa7c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:03.794790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '672f5c7b82d94040b128b42b54072266'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:03.810620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6febb3bb852434fb29d0ef3c59cfa7c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:03.811660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6febb3bb852434fb29d0ef3c59cfa7c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:03.812573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-16e86bca-10cd-4cbd-822f-042df35b0e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6febb3bb852434fb29d0ef3c59cfa7c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:03.872170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 827/1659] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:03 2022] DELETE /identity/v3/users/c6febb3bb852434fb29d0ef3c59cfa7c => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:54:03.876524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1aca9-b1fa-4f27-a44c-0fb65b9741a0 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:03.877012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1aca9-b1fa-4f27-a44c-0fb65b9741a0 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:03.877524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1aca9-b1fa-4f27-a44c-0fb65b9741a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:03.899317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cbc51bda-872b-4c59-bbd5-e449a56270c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '672f5c7b82d94040b128b42b54072266'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:03.913330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 833/1660] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:03 2022] DELETE /identity/v3/users/672f5c7b82d94040b128b42b54072266 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:03.916121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f899f0-5d7f-4d02-b143-2f54dfd19752 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:03.916430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f899f0-5d7f-4d02-b143-2f54dfd19752 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:03.916733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f899f0-5d7f-4d02-b143-2f54dfd19752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:03.926657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c3a1aca9-b1fa-4f27-a44c-0fb65b9741a0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:03.931904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c3a1aca9-b1fa-4f27-a44c-0fb65b9741a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:03.963959 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c2f899f0-5d7f-4d02-b143-2f54dfd19752 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:03.965252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c2f899f0-5d7f-4d02-b143-2f54dfd19752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:03.995510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:04.097751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 828/1661] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:04.132066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1aca9-b1fa-4f27-a44c-0fb65b9741a0 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:04.137851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c3a1aca9-b1fa-4f27-a44c-0fb65b9741a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:04.150164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 834/1662] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:04.179597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:04.222049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1aca9-b1fa-4f27-a44c-0fb65b9741a0 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:04.226765 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440687a-f2e5-45a9-be0b-3077a51a516b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:04.227519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440687a-f2e5-45a9-be0b-3077a51a516b service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:04.227802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440687a-f2e5-45a9-be0b-3077a51a516b service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:04.228077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440687a-f2e5-45a9-be0b-3077a51a516b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:04.239674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440687a-f2e5-45a9-be0b-3077a51a516b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:04.241533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440687a-f2e5-45a9-be0b-3077a51a516b service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:04.252944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f899f0-5d7f-4d02-b143-2f54dfd19752 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:04.267586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f899f0-5d7f-4d02-b143-2f54dfd19752 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:04.269765 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 829/1663] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:04.273802 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c607439-c932-4901-b3fa-4fbdd7817a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:04.274919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c607439-c932-4901-b3fa-4fbdd7817a7d service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:04.275497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c607439-c932-4901-b3fa-4fbdd7817a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:04.275931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c607439-c932-4901-b3fa-4fbdd7817a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:04.288833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c607439-c932-4901-b3fa-4fbdd7817a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:04.290824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c607439-c932-4901-b3fa-4fbdd7817a7d service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:04.305818 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 835/1664] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:04.888217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb45b17-e556-447c-9db9-2d9cb50493d4 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:04.888217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb45b17-e556-447c-9db9-2d9cb50493d4 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:04.888805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb45b17-e556-447c-9db9-2d9cb50493d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:04.920882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-1bb45b17-e556-447c-9db9-2d9cb50493d4 None None] MFA Rules not processed for user `982651a870c44a07bc4df322919e605e`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:04.922424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1bb45b17-e556-447c-9db9-2d9cb50493d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:05.113441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 830/1665] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:05.126111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c607439-c932-4901-b3fa-4fbdd7817a7d service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:05.128652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9c607439-c932-4901-b3fa-4fbdd7817a7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:05.315288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb45b17-e556-447c-9db9-2d9cb50493d4 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:05.317132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1bb45b17-e556-447c-9db9-2d9cb50493d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:05.358833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb45b17-e556-447c-9db9-2d9cb50493d4 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:05.364892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c607439-c932-4901-b3fa-4fbdd7817a7d service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:05.372542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:05.373545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:05.373770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:05.374169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] PATH_INFO: `/v3/projects/b63ffe62990842e4b20b86218bf26dc3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:05.376299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63ffe62990842e4b20b86218bf26dc3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:05.377301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: WARNING py.warnings [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 05:54:05.377301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: warnings.warn(msg) Jul 04 05:54:05.377301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: Jul 04 05:54:05.378514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:05.399885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7931b5-5764-4fe9-9ec5-46e1a07c7bea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:05.400704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7931b5-5764-4fe9-9ec5-46e1a07c7bea service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:05.401081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7931b5-5764-4fe9-9ec5-46e1a07c7bea service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:05.401589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7931b5-5764-4fe9-9ec5-46e1a07c7bea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:05.428081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7931b5-5764-4fe9-9ec5-46e1a07c7bea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:05.430541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7931b5-5764-4fe9-9ec5-46e1a07c7bea service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:05.460831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 836/1666] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:05.461953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7931b5-5764-4fe9-9ec5-46e1a07c7bea service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:05.467221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9a7931b5-5764-4fe9-9ec5-46e1a07c7bea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:05.487934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7931b5-5764-4fe9-9ec5-46e1a07c7bea service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:05.527030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63ffe62990842e4b20b86218bf26dc3', 'request_id': 'req-2d738e92-a1d4-4c7e-8801-1c43649e57af'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.552310 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63ffe62990842e4b20b86218bf26dc3', 'request_id': 'req-2d738e92-a1d4-4c7e-8801-1c43649e57af'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.567932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63ffe62990842e4b20b86218bf26dc3', 'request_id': 'req-2d738e92-a1d4-4c7e-8801-1c43649e57af'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.600969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] The token cache is being invalidate because project b63ffe62990842e4b20b86218bf26dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:05.601269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.604074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.613518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 831/1667] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:05 2022] DELETE /identity/v3/projects/b63ffe62990842e4b20b86218bf26dc3 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:05.613518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:05.616195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:05.619687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:05.620540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:05.620823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:05.621252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] PATH_INFO: `/v3/users/72cb1ae8aa5243fca51dcff137181004` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:05.636745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72cb1ae8aa5243fca51dcff137181004)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:05.638966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:05.644512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d738e92-a1d4-4c7e-8801-1c43649e57af admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:05.666071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:05.667434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:05.667721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:05.667997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] PATH_INFO: `/v3/projects/079c79c5dc674e4988dcfc371ba9901e` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:05.672348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] RBAC: Authorizing `identity:delete_project(project_id=079c79c5dc674e4988dcfc371ba9901e)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:05.683896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:05.740837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:05.761679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72cb1ae8aa5243fca51dcff137181004'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.788189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72cb1ae8aa5243fca51dcff137181004'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.789232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72cb1ae8aa5243fca51dcff137181004'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.792474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:05.808324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079c79c5dc674e4988dcfc371ba9901e', 'request_id': 'req-3aa695aa-9125-44eb-9bd8-8872b16eb88b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.812863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72cb1ae8aa5243fca51dcff137181004'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.836569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079c79c5dc674e4988dcfc371ba9901e', 'request_id': 'req-3aa695aa-9125-44eb-9bd8-8872b16eb88b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.842382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-fb29b89c-1369-4905-a970-ae769d2f9e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72cb1ae8aa5243fca51dcff137181004'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.860198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079c79c5dc674e4988dcfc371ba9901e', 'request_id': 'req-3aa695aa-9125-44eb-9bd8-8872b16eb88b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.867795 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 837/1668] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:05 2022] DELETE /identity/v3/users/72cb1ae8aa5243fca51dcff137181004 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:05.877063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4ce6-9335-402c-8be0-a5de83737793 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:05.877314 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4ce6-9335-402c-8be0-a5de83737793 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:05.877536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4ce6-9335-402c-8be0-a5de83737793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:05.912965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] The token cache is being invalidate because project 079c79c5dc674e4988dcfc371ba9901e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:05.913817 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.916710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3aa695aa-9125-44eb-9bd8-8872b16eb88b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:05.926415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 832/1669] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:05 2022] DELETE /identity/v3/projects/079c79c5dc674e4988dcfc371ba9901e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:05.928405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9475f1-c1dd-4174-a3e0-4d84fcdfc5bc None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:05.928730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9475f1-c1dd-4174-a3e0-4d84fcdfc5bc None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:05.929172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9475f1-c1dd-4174-a3e0-4d84fcdfc5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:05.952721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-309f4ce6-9335-402c-8be0-a5de83737793 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:05.955826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-309f4ce6-9335-402c-8be0-a5de83737793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:05.978647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-fa9475f1-c1dd-4174-a3e0-4d84fcdfc5bc None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:05.978647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fa9475f1-c1dd-4174-a3e0-4d84fcdfc5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:06.096062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 838/1670] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:06.097110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f4ce6-9335-402c-8be0-a5de83737793 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:06.102858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-309f4ce6-9335-402c-8be0-a5de83737793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:06.133584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f4ce6-9335-402c-8be0-a5de83737793 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:06.136511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 833/1671] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:06.137840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9660-e29e-42d4-bd6f-acee32d447b5 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:06.138118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9660-e29e-42d4-bd6f-acee32d447b5 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:06.138394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9660-e29e-42d4-bd6f-acee32d447b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:06.145930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:06.148317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:06.149409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:06.149693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] PATH_INFO: `/v3/users/960c516cd96d42b5ad531c8b7c74fa79` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:06.151995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] RBAC: Authorizing `identity:delete_user(user_id=960c516cd96d42b5ad531c8b7c74fa79)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:06.157767 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:06.183192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-986d9660-e29e-42d4-bd6f-acee32d447b5 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:06.188724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-986d9660-e29e-42d4-bd6f-acee32d447b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:06.211503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 834/1672] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:06.212450 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d9660-e29e-42d4-bd6f-acee32d447b5 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:06.215819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-986d9660-e29e-42d4-bd6f-acee32d447b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:06.268589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '960c516cd96d42b5ad531c8b7c74fa79'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:06.283917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '960c516cd96d42b5ad531c8b7c74fa79'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:06.285713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '960c516cd96d42b5ad531c8b7c74fa79'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:06.305115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '960c516cd96d42b5ad531c8b7c74fa79'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:06.325822 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '960c516cd96d42b5ad531c8b7c74fa79'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:06.343224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 839/1673] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:06 2022] DELETE /identity/v3/users/960c516cd96d42b5ad531c8b7c74fa79 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:06.344773 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:06.346487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:06.349129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d9660-e29e-42d4-bd6f-acee32d447b5 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:06.395750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:06.493224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb4bf1-89e2-4408-911c-0de2bd3c5aab admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:06.502624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753868c-4552-4e30-8955-5b3bc790e3b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:06.503695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753868c-4552-4e30-8955-5b3bc790e3b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:06.506577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e95df-c255-40e6-8c0b-9c90522bff08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:06.507925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e95df-c255-40e6-8c0b-9c90522bff08 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:06.508323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e95df-c255-40e6-8c0b-9c90522bff08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:06.509037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e95df-c255-40e6-8c0b-9c90522bff08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:06.509928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753868c-4552-4e30-8955-5b3bc790e3b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:06.510419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753868c-4552-4e30-8955-5b3bc790e3b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:06.510648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e95df-c255-40e6-8c0b-9c90522bff08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:06.510979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e95df-c255-40e6-8c0b-9c90522bff08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:06.515608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e95df-c255-40e6-8c0b-9c90522bff08 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:06.521275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6753868c-4552-4e30-8955-5b3bc790e3b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:06.541878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 840/1674] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:06.545935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e95df-c255-40e6-8c0b-9c90522bff08 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:06.570223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e95df-c255-40e6-8c0b-9c90522bff08 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:06.575377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753868c-4552-4e30-8955-5b3bc790e3b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:06.577215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753868c-4552-4e30-8955-5b3bc790e3b8 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:06.589121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bda643-4754-41c0-adfd-29b0aac54ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:06.591665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bda643-4754-41c0-adfd-29b0aac54ace admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:06.592598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bda643-4754-41c0-adfd-29b0aac54ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:06.592983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bda643-4754-41c0-adfd-29b0aac54ace admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:06.596408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bda643-4754-41c0-adfd-29b0aac54ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:06.601071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bda643-4754-41c0-adfd-29b0aac54ace admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:06.601287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 835/1675] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:06.610564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002d7ee-c8bc-4d4e-aea9-6182729251b2 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:06.610977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002d7ee-c8bc-4d4e-aea9-6182729251b2 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:06.611291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002d7ee-c8bc-4d4e-aea9-6182729251b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:06.668556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 841/1676] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:06 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:54:06.672799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bda643-4754-41c0-adfd-29b0aac54ace admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:06.740060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c002d7ee-c8bc-4d4e-aea9-6182729251b2 None None] MFA Rules not processed for user `3e05d19530a74e05a11be774102b425a`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:06.742578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c002d7ee-c8bc-4d4e-aea9-6182729251b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:06.779802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:06.839570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bda643-4754-41c0-adfd-29b0aac54ace admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:06.851672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a2ca3-42a7-480d-9308-de4a23c2e74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:06.853691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a2ca3-42a7-480d-9308-de4a23c2e74b admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:06.856103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a2ca3-42a7-480d-9308-de4a23c2e74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:06.856395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a2ca3-42a7-480d-9308-de4a23c2e74b admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:06.858639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a2ca3-42a7-480d-9308-de4a23c2e74b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:06.862909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a2ca3-42a7-480d-9308-de4a23c2e74b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:06.912083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 842/1677] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:06 2022] POST /identity/v3/users => generated 284 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:54:06.913770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a2ca3-42a7-480d-9308-de4a23c2e74b admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:06.936324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a2ca3-42a7-480d-9308-de4a23c2e74b admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:06.950008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9cbcb-8b96-46a2-9587-4ce40cb0ab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:06.951781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9cbcb-8b96-46a2-9587-4ce40cb0ab32 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:06.952081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9cbcb-8b96-46a2-9587-4ce40cb0ab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:06.952352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9cbcb-8b96-46a2-9587-4ce40cb0ab32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:06.953967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9cbcb-8b96-46a2-9587-4ce40cb0ab32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:06.955642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9cbcb-8b96-46a2-9587-4ce40cb0ab32 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:06.971643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 836/1678] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:06.974248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9186b3a-7746-435b-9c82-c0abbc5cf309 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:06.974543 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9186b3a-7746-435b-9c82-c0abbc5cf309 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:06.974826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9186b3a-7746-435b-9c82-c0abbc5cf309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:06.992456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 843/1679] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:07.008927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9cbcb-8b96-46a2-9587-4ce40cb0ab32 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:07.049670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9cbcb-8b96-46a2-9587-4ce40cb0ab32 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:07.071144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bcc3e-9092-4d15-a238-184ad48fcb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:07.075454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bcc3e-9092-4d15-a238-184ad48fcb03 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:07.076897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bcc3e-9092-4d15-a238-184ad48fcb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:07.077689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bcc3e-9092-4d15-a238-184ad48fcb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:07.080980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bcc3e-9092-4d15-a238-184ad48fcb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:07.083977 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bcc3e-9092-4d15-a238-184ad48fcb03 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:07.134629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 844/1680] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:07.137633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bcc3e-9092-4d15-a238-184ad48fcb03 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:07.160544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bcc3e-9092-4d15-a238-184ad48fcb03 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:07.176729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f6a6a-21df-4c45-9e47-41b4d0448f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:07.180042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f6a6a-21df-4c45-9e47-41b4d0448f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:07.180455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f6a6a-21df-4c45-9e47-41b4d0448f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:07.180834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f6a6a-21df-4c45-9e47-41b4d0448f13 admin admin] PATH_INFO: `/v3/projects/0918524e02a74f1cbcc89142d4d8d012/users/9779bb6318d046dc8caea86b3efbc31a/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:07.225635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f6a6a-21df-4c45-9e47-41b4d0448f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0918524e02a74f1cbcc89142d4d8d012, user_id=9779bb6318d046dc8caea86b3efbc31a, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:07.229676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f6a6a-21df-4c45-9e47-41b4d0448f13 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:07.271777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-e9186b3a-7746-435b-9c82-c0abbc5cf309 None None] MFA Rules not processed for user `3e05d19530a74e05a11be774102b425a`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:07.273395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 845/1681] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:07 2022] PUT /identity/v3/projects/0918524e02a74f1cbcc89142d4d8d012/users/9779bb6318d046dc8caea86b3efbc31a/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:07.273667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e9186b3a-7746-435b-9c82-c0abbc5cf309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:07.278471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd8c80-2292-48be-946b-6cf40c34c4fc None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:07.278937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd8c80-2292-48be-946b-6cf40c34c4fc None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:07.279396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd8c80-2292-48be-946b-6cf40c34c4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:07.358309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-7abd8c80-2292-48be-946b-6cf40c34c4fc None None] MFA Rules not processed for user `9779bb6318d046dc8caea86b3efbc31a`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:07.359936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7abd8c80-2292-48be-946b-6cf40c34c4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:07.423838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:07.428336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 837/1682] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:07.441873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9186b3a-7746-435b-9c82-c0abbc5cf309 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:07.443761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e9186b3a-7746-435b-9c82-c0abbc5cf309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:07.542557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 846/1683] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:07.543748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd8c80-2292-48be-946b-6cf40c34c4fc None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:07.549179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7abd8c80-2292-48be-946b-6cf40c34c4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:07.630667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9186b3a-7746-435b-9c82-c0abbc5cf309 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:07.639188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb4f85-d536-48f8-8712-0962839b35b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:07.640102 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb4f85-d536-48f8-8712-0962839b35b9 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:07.640389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb4f85-d536-48f8-8712-0962839b35b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:07.640663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb4f85-d536-48f8-8712-0962839b35b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:07.658138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb4f85-d536-48f8-8712-0962839b35b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:07.660141 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb4f85-d536-48f8-8712-0962839b35b9 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:07.679820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd8c80-2292-48be-946b-6cf40c34c4fc None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:07.685558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 838/1684] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:07.698796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f465d9-fa59-4536-8766-f8d0e6f02f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:07.701381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f465d9-fa59-4536-8766-f8d0e6f02f64 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:07.701381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f465d9-fa59-4536-8766-f8d0e6f02f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:07.701381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f465d9-fa59-4536-8766-f8d0e6f02f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:07.741923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f465d9-fa59-4536-8766-f8d0e6f02f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:07.744975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f465d9-fa59-4536-8766-f8d0e6f02f64 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:07.784420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 847/1685] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:08.684267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb4f85-d536-48f8-8712-0962839b35b9 service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:08.685721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-acfb4f85-d536-48f8-8712-0962839b35b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:08.837255 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:08.882285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb4f85-d536-48f8-8712-0962839b35b9 service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:08.891071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:08.892676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:08.895958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:08.897843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] PATH_INFO: `/v3/projects/fdd60ea786604e6da727ea31ed373c93` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:08.906179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd60ea786604e6da727ea31ed373c93)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:08.908736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:09.000768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f465d9-fa59-4536-8766-f8d0e6f02f64 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:09.005652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-26f465d9-fa59-4536-8766-f8d0e6f02f64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:09.024904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd60ea786604e6da727ea31ed373c93', 'request_id': 'req-c791644b-66ff-4c04-835e-72b3b12dbb17'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:09.035727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd60ea786604e6da727ea31ed373c93', 'request_id': 'req-c791644b-66ff-4c04-835e-72b3b12dbb17'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:09.048991 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f465d9-fa59-4536-8766-f8d0e6f02f64 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:09.053997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd60ea786604e6da727ea31ed373c93', 'request_id': 'req-c791644b-66ff-4c04-835e-72b3b12dbb17'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:09.105479 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] The token cache is being invalidate because project fdd60ea786604e6da727ea31ed373c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:09.105821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:09.107687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c791644b-66ff-4c04-835e-72b3b12dbb17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:09.111038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 839/1686] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:08 2022] DELETE /identity/v3/projects/fdd60ea786604e6da727ea31ed373c93 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:09.123890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fd4f0-2e36-4b1a-af73-86619f2339c8 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:09.124905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fd4f0-2e36-4b1a-af73-86619f2339c8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:09.125551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fd4f0-2e36-4b1a-af73-86619f2339c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:09.178213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-0b5fd4f0-2e36-4b1a-af73-86619f2339c8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:09.179687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0b5fd4f0-2e36-4b1a-af73-86619f2339c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:09.378524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:09.380231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:09.384931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:09.385830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] PATH_INFO: `/v3/projects/cbb322e1350d44e496c53ffc9b869d7a` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:09.392842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb322e1350d44e496c53ffc9b869d7a)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:09.414414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:09.505199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 840/1687] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:09.505199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5ec9d-5e23-4f5e-af5f-5e815f88f157 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:09.505199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5ec9d-5e23-4f5e-af5f-5e815f88f157 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:09.505199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5ec9d-5e23-4f5e-af5f-5e815f88f157 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:09.514289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 841/1688] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jul 4 05:54:09 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:54:09.516193 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e41975-9aea-4c81-b781-634ca99fde40 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:09.525330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e41975-9aea-4c81-b781-634ca99fde40 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:09.525797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e41975-9aea-4c81-b781-634ca99fde40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:09.595407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-19e41975-9aea-4c81-b781-634ca99fde40 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:09.602933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-19e41975-9aea-4c81-b781-634ca99fde40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:09.633576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb322e1350d44e496c53ffc9b869d7a', 'request_id': 'req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:09.657166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb322e1350d44e496c53ffc9b869d7a', 'request_id': 'req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:09.706817 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb322e1350d44e496c53ffc9b869d7a', 'request_id': 'req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:09.739829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:09.750698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] The token cache is being invalidate because project cbb322e1350d44e496c53ffc9b869d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:09.751327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:09.754144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-34117f3e-84a4-4c4f-a2a4-48fdcc2566a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:09.761313 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 848/1689] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:08 2022] DELETE /identity/v3/projects/cbb322e1350d44e496c53ffc9b869d7a => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:09.778650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5325c-9cf9-49f9-931d-4d9f885cf5a0 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:09.779308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5325c-9cf9-49f9-931d-4d9f885cf5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:09.779774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5325c-9cf9-49f9-931d-4d9f885cf5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:09.898550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 842/1690] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:09.902050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ed907-33d5-4e9c-a5ff-f9a91c1c1319 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:09.902575 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ed907-33d5-4e9c-a5ff-f9a91c1c1319 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:09.904655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ed907-33d5-4e9c-a5ff-f9a91c1c1319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:09.929346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-0ea5325c-9cf9-49f9-931d-4d9f885cf5a0 None None] MFA Rules not processed for user `817b1a026f4b443daea514338ad8531a`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:09.931865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0ea5325c-9cf9-49f9-931d-4d9f885cf5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:09.967537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-579ed907-33d5-4e9c-a5ff-f9a91c1c1319 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:09.974923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-579ed907-33d5-4e9c-a5ff-f9a91c1c1319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:09.996455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 843/1691] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:09.997742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ed907-33d5-4e9c-a5ff-f9a91c1c1319 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:10.022290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:10.032623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ed907-33d5-4e9c-a5ff-f9a91c1c1319 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:10.052879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf284e3f-e3e3-4525-9b44-48b28cae5a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:10.053856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf284e3f-e3e3-4525-9b44-48b28cae5a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:10.054255 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf284e3f-e3e3-4525-9b44-48b28cae5a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:10.054631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf284e3f-e3e3-4525-9b44-48b28cae5a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:10.055645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf284e3f-e3e3-4525-9b44-48b28cae5a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:10.056099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf284e3f-e3e3-4525-9b44-48b28cae5a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:10.060307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf284e3f-e3e3-4525-9b44-48b28cae5a94 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:10.105675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 844/1692] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:10.108029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b963af-2dcc-4092-8de3-e7f57fc580b7 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:10.110384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b963af-2dcc-4092-8de3-e7f57fc580b7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:10.110780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b963af-2dcc-4092-8de3-e7f57fc580b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:10.130435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 849/1693] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 05:54:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:10.138829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5325c-9cf9-49f9-931d-4d9f885cf5a0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:10.175097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5325c-9cf9-49f9-931d-4d9f885cf5a0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:10.182895 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-49b963af-2dcc-4092-8de3-e7f57fc580b7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:10.185180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-49b963af-2dcc-4092-8de3-e7f57fc580b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:10.198592 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92368f-e760-431a-a292-b7f5ae184218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:10.200364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92368f-e760-431a-a292-b7f5ae184218 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:10.200658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92368f-e760-431a-a292-b7f5ae184218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:10.200945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92368f-e760-431a-a292-b7f5ae184218 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:10.206509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92368f-e760-431a-a292-b7f5ae184218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:10.214538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92368f-e760-431a-a292-b7f5ae184218 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:10.224903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 845/1694] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:10.225864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b963af-2dcc-4092-8de3-e7f57fc580b7 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:10.232287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-49b963af-2dcc-4092-8de3-e7f57fc580b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:10.271843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b963af-2dcc-4092-8de3-e7f57fc580b7 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:10.295500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e46db-3098-4625-8bd4-ab0152727c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:10.296669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e46db-3098-4625-8bd4-ab0152727c4d service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:10.298442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e46db-3098-4625-8bd4-ab0152727c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:10.304103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e46db-3098-4625-8bd4-ab0152727c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:10.325646 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e46db-3098-4625-8bd4-ab0152727c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:10.327208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e46db-3098-4625-8bd4-ab0152727c4d service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:10.329025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 850/1695] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:10 2022] POST /identity/v3/projects => generated 403 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:10.332708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92368f-e760-431a-a292-b7f5ae184218 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:10.371506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 846/1696] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:10.380315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e46db-3098-4625-8bd4-ab0152727c4d service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:10.418687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e46db-3098-4625-8bd4-ab0152727c4d service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:10.585337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92368f-e760-431a-a292-b7f5ae184218 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:10.625191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d94292-539f-475f-a1f1-4d81ab9d07cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:10.625786 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d94292-539f-475f-a1f1-4d81ab9d07cf admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:10.626079 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d94292-539f-475f-a1f1-4d81ab9d07cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:10.626354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d94292-539f-475f-a1f1-4d81ab9d07cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:10.633884 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d94292-539f-475f-a1f1-4d81ab9d07cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:10.634213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d94292-539f-475f-a1f1-4d81ab9d07cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:10.636460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d94292-539f-475f-a1f1-4d81ab9d07cf admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:10.652449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 851/1697] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:10.658285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d878d5-6d74-4e8e-a736-86cfdb4e6824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:10.658778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d94292-539f-475f-a1f1-4d81ab9d07cf admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:10.658997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d878d5-6d74-4e8e-a736-86cfdb4e6824 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:10.664389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d878d5-6d74-4e8e-a736-86cfdb4e6824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:10.664784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d878d5-6d74-4e8e-a736-86cfdb4e6824 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:10.665550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d878d5-6d74-4e8e-a736-86cfdb4e6824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:10.670851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d878d5-6d74-4e8e-a736-86cfdb4e6824 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:10.701212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d94292-539f-475f-a1f1-4d81ab9d07cf admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:10.720082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61d6e2-1577-45a0-8bdd-21e74b5df131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:10.721731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61d6e2-1577-45a0-8bdd-21e74b5df131 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:10.725273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61d6e2-1577-45a0-8bdd-21e74b5df131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:10.730464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61d6e2-1577-45a0-8bdd-21e74b5df131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:10.734570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61d6e2-1577-45a0-8bdd-21e74b5df131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:10.736942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61d6e2-1577-45a0-8bdd-21e74b5df131 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:10.744919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 847/1698] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:10 2022] POST /identity/v3/users => generated 302 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:10.751783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d878d5-6d74-4e8e-a736-86cfdb4e6824 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:10.786093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d878d5-6d74-4e8e-a736-86cfdb4e6824 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:10.798116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba677b81-d83e-44b7-b487-1f4b75ec0fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:10.798833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba677b81-d83e-44b7-b487-1f4b75ec0fed admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:10.799181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba677b81-d83e-44b7-b487-1f4b75ec0fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:10.799426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba677b81-d83e-44b7-b487-1f4b75ec0fed admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:10.801167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba677b81-d83e-44b7-b487-1f4b75ec0fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:10.808687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba677b81-d83e-44b7-b487-1f4b75ec0fed admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:10.847005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 852/1700] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:10 2022] POST /identity/v3/projects => generated 379 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:10.847320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 848/1700] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:10.847406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61d6e2-1577-45a0-8bdd-21e74b5df131 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:10.848202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba677b81-d83e-44b7-b487-1f4b75ec0fed admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:10.876180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba677b81-d83e-44b7-b487-1f4b75ec0fed admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:11.009308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:11.040102 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9f0bf-e1ef-4d3d-ad2f-16b143b6ea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:11.040824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9f0bf-e1ef-4d3d-ad2f-16b143b6ea33 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:11.041167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9f0bf-e1ef-4d3d-ad2f-16b143b6ea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:11.041451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9f0bf-e1ef-4d3d-ad2f-16b143b6ea33 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:11.042511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9f0bf-e1ef-4d3d-ad2f-16b143b6ea33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:11.044380 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9f0bf-e1ef-4d3d-ad2f-16b143b6ea33 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:11.063496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61d6e2-1577-45a0-8bdd-21e74b5df131 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:11.073775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec71a13-763e-484e-91cc-dbcb0f43ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:11.077072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec71a13-763e-484e-91cc-dbcb0f43ce71 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:11.077633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec71a13-763e-484e-91cc-dbcb0f43ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:11.078107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec71a13-763e-484e-91cc-dbcb0f43ce71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:11.081164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec71a13-763e-484e-91cc-dbcb0f43ce71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:11.083008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec71a13-763e-484e-91cc-dbcb0f43ce71 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:11.110186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 849/1701] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:10 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:11.113490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 853/1702] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:11.117350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9f0bf-e1ef-4d3d-ad2f-16b143b6ea33 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:11.117541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec71a13-763e-484e-91cc-dbcb0f43ce71 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:11.137899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9f0bf-e1ef-4d3d-ad2f-16b143b6ea33 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:11.144560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98b7b4-055e-4f1b-a0ce-1ae616344b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:11.145975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98b7b4-055e-4f1b-a0ce-1ae616344b94 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:11.145975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98b7b4-055e-4f1b-a0ce-1ae616344b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:11.145975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98b7b4-055e-4f1b-a0ce-1ae616344b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:11.146720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98b7b4-055e-4f1b-a0ce-1ae616344b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:11.149088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98b7b4-055e-4f1b-a0ce-1ae616344b94 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:11.150131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec71a13-763e-484e-91cc-dbcb0f43ce71 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:11.181358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 850/1703] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:11.185111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f1026-a4c8-4843-ac70-b50fd0288bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:11.185399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98b7b4-055e-4f1b-a0ce-1ae616344b94 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:11.186041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f1026-a4c8-4843-ac70-b50fd0288bac admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:11.186472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f1026-a4c8-4843-ac70-b50fd0288bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:11.186990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f1026-a4c8-4843-ac70-b50fd0288bac admin admin] PATH_INFO: `/v3/projects/8a928afbe8c84761b9e7f351cee6bdd5/users/ebcc99ee16da4f928c8372e8d666a307/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:11.208725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98b7b4-055e-4f1b-a0ce-1ae616344b94 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:11.236530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abcf43-213b-4e5b-be84-0ac5ce0e4094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:11.237246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abcf43-213b-4e5b-be84-0ac5ce0e4094 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:11.237548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abcf43-213b-4e5b-be84-0ac5ce0e4094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:11.237804 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abcf43-213b-4e5b-be84-0ac5ce0e4094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:11.247532 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abcf43-213b-4e5b-be84-0ac5ce0e4094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:11.251429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abcf43-213b-4e5b-be84-0ac5ce0e4094 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:11.260333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f1026-a4c8-4843-ac70-b50fd0288bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a928afbe8c84761b9e7f351cee6bdd5, user_id=ebcc99ee16da4f928c8372e8d666a307, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:11.263029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f1026-a4c8-4843-ac70-b50fd0288bac admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:11.307615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 851/1704] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:11.310418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abcf43-213b-4e5b-be84-0ac5ce0e4094 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:11.321473 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 854/1705] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:11 2022] PUT /identity/v3/projects/8a928afbe8c84761b9e7f351cee6bdd5/users/ebcc99ee16da4f928c8372e8d666a307/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:11.326498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b8c48-653a-4e6b-949e-a429a177b07f None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:11.327045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b8c48-653a-4e6b-949e-a429a177b07f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:11.327663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b8c48-653a-4e6b-949e-a429a177b07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:11.359031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abcf43-213b-4e5b-be84-0ac5ce0e4094 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:11.373288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6d7b8c48-653a-4e6b-949e-a429a177b07f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:11.380453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6d7b8c48-653a-4e6b-949e-a429a177b07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:11.388489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:11.516684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 855/1706] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:11.584603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae3120-8b0c-487e-996c-caa71934a5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:11.585547 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae3120-8b0c-487e-996c-caa71934a5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:11.585837 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae3120-8b0c-487e-996c-caa71934a5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:11.586116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae3120-8b0c-487e-996c-caa71934a5d7 admin admin] PATH_INFO: `/v3/projects/512f151955cb43a1bae07da95b4e52f2/users/4432520048d04008aa5d930828dbae2a/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:11.640480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae3120-8b0c-487e-996c-caa71934a5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=512f151955cb43a1bae07da95b4e52f2, user_id=4432520048d04008aa5d930828dbae2a, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:11.643196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae3120-8b0c-487e-996c-caa71934a5d7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:11.648042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b8c48-653a-4e6b-949e-a429a177b07f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:11.685692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 852/1707] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:11 2022] PUT /identity/v3/projects/512f151955cb43a1bae07da95b4e52f2/users/4432520048d04008aa5d930828dbae2a/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:11.696258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b84ec-43df-4eb7-a5d1-ab5d59040acd None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:11.696544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b84ec-43df-4eb7-a5d1-ab5d59040acd None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:11.696820 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b84ec-43df-4eb7-a5d1-ab5d59040acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:11.806087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ec9b84ec-43df-4eb7-a5d1-ab5d59040acd None None] MFA Rules not processed for user `4432520048d04008aa5d930828dbae2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:11.807415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ec9b84ec-43df-4eb7-a5d1-ab5d59040acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:11.817300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b8c48-653a-4e6b-949e-a429a177b07f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:12.030521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 853/1708] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:12.033337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b84ec-43df-4eb7-a5d1-ab5d59040acd None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:12.088571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f1fe4-330e-41ea-9627-d1a8161b2eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:12.089273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f1fe4-330e-41ea-9627-d1a8161b2eca service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:12.095408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f1fe4-330e-41ea-9627-d1a8161b2eca service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:12.095408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f1fe4-330e-41ea-9627-d1a8161b2eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:12.165579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:12.204551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b84ec-43df-4eb7-a5d1-ab5d59040acd None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:12.217138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b9198-fb2f-4913-80ae-8fbf539fa8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:12.217913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b9198-fb2f-4913-80ae-8fbf539fa8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:12.218181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b9198-fb2f-4913-80ae-8fbf539fa8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:12.218453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b9198-fb2f-4913-80ae-8fbf539fa8e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:12.219325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b9198-fb2f-4913-80ae-8fbf539fa8e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:12.222085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b9198-fb2f-4913-80ae-8fbf539fa8e0 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:12.277557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f1fe4-330e-41ea-9627-d1a8161b2eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:12.281337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f1fe4-330e-41ea-9627-d1a8161b2eca service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:12.325078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 854/1709] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:12 2022] POST /identity/v3/projects => generated 379 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:12.325078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a8d38-7d80-4385-9890-0590a234b5e9 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:12.325345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 856/1710] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:12.325427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a8d38-7d80-4385-9890-0590a234b5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:12.325427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a8d38-7d80-4385-9890-0590a234b5e9 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:12.327522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f1fe4-330e-41ea-9627-d1a8161b2eca service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:12.333314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 855/1711] 10.222.0.42 () {54 vars in 960 bytes} [Mon Jul 4 05:54:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:54:12.334819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca02841-e043-47e3-b466-ea48d2af94b0 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:12.335354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca02841-e043-47e3-b466-ea48d2af94b0 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:12.335491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca02841-e043-47e3-b466-ea48d2af94b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:12.366028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f1fe4-330e-41ea-9627-d1a8161b2eca service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:12.376028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-cca02841-e043-47e3-b466-ea48d2af94b0 None None] MFA Rules not processed for user `486de932d1db422ba2c3d84142f9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:12.377524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cca02841-e043-47e3-b466-ea48d2af94b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:12.400619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:12.548261 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 856/1712] 10.222.0.42 () {58 vars in 1057 bytes} [Mon Jul 4 05:54:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:12.562985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b92fc-07f5-4a99-aad2-5a6511e9bab9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:12.564301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b92fc-07f5-4a99-aad2-5a6511e9bab9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:12.565344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b92fc-07f5-4a99-aad2-5a6511e9bab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:12.575145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48606b6c-6b35-4596-9372-6523f25d9bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:12.575888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48606b6c-6b35-4596-9372-6523f25d9bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:12.576173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48606b6c-6b35-4596-9372-6523f25d9bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:12.581606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48606b6c-6b35-4596-9372-6523f25d9bbf admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:12.586639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48606b6c-6b35-4596-9372-6523f25d9bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:12.591119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48606b6c-6b35-4596-9372-6523f25d9bbf admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:12.684044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-1f6b92fc-07f5-4a99-aad2-5a6511e9bab9 None None] MFA Rules not processed for user `32703b5451cf48b09966a7412b747099`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:12.687782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1f6b92fc-07f5-4a99-aad2-5a6511e9bab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:12.722864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 857/1713] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:12 2022] POST /identity/v3/users => generated 296 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:12.725083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48606b6c-6b35-4596-9372-6523f25d9bbf admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:12.780956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48606b6c-6b35-4596-9372-6523f25d9bbf admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:12.818993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83810-4ef0-4bd5-b540-575a76256e24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:12.819715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83810-4ef0-4bd5-b540-575a76256e24 service placement] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:12.819998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83810-4ef0-4bd5-b540-575a76256e24 service placement] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:12.820282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83810-4ef0-4bd5-b540-575a76256e24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:12.827426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 857/1714] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:12.827426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b92fc-07f5-4a99-aad2-5a6511e9bab9 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:12.832952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1ff83810-4ef0-4bd5-b540-575a76256e24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:12.846203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b92fc-07f5-4a99-aad2-5a6511e9bab9 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:12.855387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c3103-3ce8-4280-818f-6acf24e71abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:12.856084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c3103-3ce8-4280-818f-6acf24e71abe admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:12.856501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c3103-3ce8-4280-818f-6acf24e71abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:12.856777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c3103-3ce8-4280-818f-6acf24e71abe admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:12.857518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c3103-3ce8-4280-818f-6acf24e71abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:12.860366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c3103-3ce8-4280-818f-6acf24e71abe admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:12.862452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff83810-4ef0-4bd5-b540-575a76256e24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:12.864155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff83810-4ef0-4bd5-b540-575a76256e24 service placement] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:12.890192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 858/1715] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:12.890192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c3103-3ce8-4280-818f-6acf24e71abe admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:12.894475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-194c3103-3ce8-4280-818f-6acf24e71abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:12.906291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 858/1716] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:12.906291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83810-4ef0-4bd5-b540-575a76256e24 service placement] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:12.937480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83810-4ef0-4bd5-b540-575a76256e24 service placement] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:12.945367 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d3e4a-71d5-44b6-8ce6-64d042638125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:12.957097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d3e4a-71d5-44b6-8ce6-64d042638125 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:12.957853 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d3e4a-71d5-44b6-8ce6-64d042638125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:12.960394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d3e4a-71d5-44b6-8ce6-64d042638125 admin admin] PATH_INFO: `/v3/projects/6a2311b2aa2340c5b5d2eab17913b411/users/e7d6e291cd914d5a826118cc7d3cd484/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:12.996671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d3e4a-71d5-44b6-8ce6-64d042638125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a2311b2aa2340c5b5d2eab17913b411, user_id=e7d6e291cd914d5a826118cc7d3cd484, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:13.003876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d3e4a-71d5-44b6-8ce6-64d042638125 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:13.044754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 859/1717] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:12 2022] PUT /identity/v3/projects/6a2311b2aa2340c5b5d2eab17913b411/users/e7d6e291cd914d5a826118cc7d3cd484/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:13.046553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d3e4a-71d5-44b6-8ce6-64d042638125 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:13.173565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c3103-3ce8-4280-818f-6acf24e71abe admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:13.175721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d3e4a-71d5-44b6-8ce6-64d042638125 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:13.184359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7975299-93d2-4b4c-9663-f90837ffc81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:13.185181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7975299-93d2-4b4c-9663-f90837ffc81c admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:13.186544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7975299-93d2-4b4c-9663-f90837ffc81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:13.187198 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7975299-93d2-4b4c-9663-f90837ffc81c admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:13.188574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7975299-93d2-4b4c-9663-f90837ffc81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:13.191043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7975299-93d2-4b4c-9663-f90837ffc81c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:13.214943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:13.215759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 860/1718] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:13.217318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fe497-e1ae-4a64-a4b0-6d10ffb2d4b6 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:13.217808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fe497-e1ae-4a64-a4b0-6d10ffb2d4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:13.219291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fe497-e1ae-4a64-a4b0-6d10ffb2d4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:13.252461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:13.284991 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-826fe497-e1ae-4a64-a4b0-6d10ffb2d4b6 None None] MFA Rules not processed for user `e367c76dfa7f4d2791e60ce148f352c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:13.286648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-826fe497-e1ae-4a64-a4b0-6d10ffb2d4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:13.348191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883a841-d719-491e-832f-4f8193106551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:13.350926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883a841-d719-491e-832f-4f8193106551 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:13.351217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883a841-d719-491e-832f-4f8193106551 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:13.351546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883a841-d719-491e-832f-4f8193106551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:13.502941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 861/1719] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:13.511781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fe497-e1ae-4a64-a4b0-6d10ffb2d4b6 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:13.524495 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fe497-e1ae-4a64-a4b0-6d10ffb2d4b6 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:13.529930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc54cad-3e11-49eb-be6c-30a66227028e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:13.530981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc54cad-3e11-49eb-be6c-30a66227028e admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:13.531364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc54cad-3e11-49eb-be6c-30a66227028e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:13.531709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc54cad-3e11-49eb-be6c-30a66227028e admin admin] PATH_INFO: `/v3/projects/6a2311b2aa2340c5b5d2eab17913b411/users/e7d6e291cd914d5a826118cc7d3cd484/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:13.533871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc54cad-3e11-49eb-be6c-30a66227028e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a2311b2aa2340c5b5d2eab17913b411, user_id=e7d6e291cd914d5a826118cc7d3cd484, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:13.534306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883a841-d719-491e-832f-4f8193106551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:13.536757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc54cad-3e11-49eb-be6c-30a66227028e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:13.539109 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883a841-d719-491e-832f-4f8193106551 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:13.563303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 862/1720] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:13 2022] PUT /identity/v3/projects/6a2311b2aa2340c5b5d2eab17913b411/users/e7d6e291cd914d5a826118cc7d3cd484/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:13.565399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a4b99-6f4d-4bd8-baee-a3ec49bae2c5 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:13.565814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a4b99-6f4d-4bd8-baee-a3ec49bae2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:13.566162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a4b99-6f4d-4bd8-baee-a3ec49bae2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:13.567841 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 859/1721] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:13.570438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883a841-d719-491e-832f-4f8193106551 service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:13.573698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4883a841-d719-491e-832f-4f8193106551 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:13.587844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-8e8a4b99-6f4d-4bd8-baee-a3ec49bae2c5 None None] MFA Rules not processed for user `982651a870c44a07bc4df322919e605e`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:13.590172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8e8a4b99-6f4d-4bd8-baee-a3ec49bae2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:13.610652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883a841-d719-491e-832f-4f8193106551 service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:13.766475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 863/1722] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:13.769678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3923729-7dfa-4974-b22c-3d6289e3ccf3 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:13.769970 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3923729-7dfa-4974-b22c-3d6289e3ccf3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:13.770248 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3923729-7dfa-4974-b22c-3d6289e3ccf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:13.783513 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529dc5e0-8531-44ef-88cc-fc0cdbd7aabc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:13.784406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529dc5e0-8531-44ef-88cc-fc0cdbd7aabc service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:13.784843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529dc5e0-8531-44ef-88cc-fc0cdbd7aabc service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:13.785405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529dc5e0-8531-44ef-88cc-fc0cdbd7aabc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:13.847987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-e3923729-7dfa-4974-b22c-3d6289e3ccf3 None None] MFA Rules not processed for user `e7d6e291cd914d5a826118cc7d3cd484`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:13.851453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e3923729-7dfa-4974-b22c-3d6289e3ccf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:14.008377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529dc5e0-8531-44ef-88cc-fc0cdbd7aabc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:14.010836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529dc5e0-8531-44ef-88cc-fc0cdbd7aabc service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:14.057703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 860/1723] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:14.226145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 864/1724] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:14.234808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529dc5e0-8531-44ef-88cc-fc0cdbd7aabc service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:14.236494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-529dc5e0-8531-44ef-88cc-fc0cdbd7aabc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:14.242782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:14.432750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529dc5e0-8531-44ef-88cc-fc0cdbd7aabc service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:14.450307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070738a-37ba-4463-a6ea-20cd54fa9598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:14.453019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070738a-37ba-4463-a6ea-20cd54fa9598 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:14.463549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070738a-37ba-4463-a6ea-20cd54fa9598 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:14.463899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070738a-37ba-4463-a6ea-20cd54fa9598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:14.524173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070738a-37ba-4463-a6ea-20cd54fa9598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:14.524173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070738a-37ba-4463-a6ea-20cd54fa9598 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:14.574216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 861/1725] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:14.768673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3923729-7dfa-4974-b22c-3d6289e3ccf3 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:14.783178 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3923729-7dfa-4974-b22c-3d6289e3ccf3 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:14.791833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01320d42-fca5-4837-8c04-5800ed99ad4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:14.794481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01320d42-fca5-4837-8c04-5800ed99ad4d service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:14.794979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01320d42-fca5-4837-8c04-5800ed99ad4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:14.795388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01320d42-fca5-4837-8c04-5800ed99ad4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:14.840359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:15.049568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01320d42-fca5-4837-8c04-5800ed99ad4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:15.052686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01320d42-fca5-4837-8c04-5800ed99ad4d service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:15.084575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 865/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:15.144703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a4184-7134-4346-8f63-c9aea986e97d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:15.145210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a4184-7134-4346-8f63-c9aea986e97d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:15.145676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a4184-7134-4346-8f63-c9aea986e97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:15.273795 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-774a4184-7134-4346-8f63-c9aea986e97d None None] MFA Rules not processed for user `ebcc99ee16da4f928c8372e8d666a307`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:15.281008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-774a4184-7134-4346-8f63-c9aea986e97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:15.481875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93404e1b-5d2e-429c-b379-dd0ca02db603 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:15.482793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93404e1b-5d2e-429c-b379-dd0ca02db603 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:15.483311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93404e1b-5d2e-429c-b379-dd0ca02db603 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:15.485874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 866/1727] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jul 4 05:54:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:54:15.496044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cc848-37a2-431f-8589-f2dfd4e25bd4 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:15.496603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cc848-37a2-431f-8589-f2dfd4e25bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:15.497025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cc848-37a2-431f-8589-f2dfd4e25bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:15.553767 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d34cc848-37a2-431f-8589-f2dfd4e25bd4 None None] MFA Rules not processed for user `817b1a026f4b443daea514338ad8531a`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:15.555076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d34cc848-37a2-431f-8589-f2dfd4e25bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:15.568396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 862/1728] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:15.569444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a4184-7134-4346-8f63-c9aea986e97d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:15.724995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 867/1729] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 05:54:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 04 05:54:15.726349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34cc848-37a2-431f-8589-f2dfd4e25bd4 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:15.729418 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d34cc848-37a2-431f-8589-f2dfd4e25bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:15.750861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34cc848-37a2-431f-8589-f2dfd4e25bd4 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:15.754683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a4184-7134-4346-8f63-c9aea986e97d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:15.760657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e1b74-f482-46e6-aeb9-05276366865d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:15.761582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e1b74-f482-46e6-aeb9-05276366865d service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:15.762611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e1b74-f482-46e6-aeb9-05276366865d service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:15.763557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e1b74-f482-46e6-aeb9-05276366865d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:15.767218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cc900-454a-4cb2-b366-80b2eb3d81a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:15.768670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cc900-454a-4cb2-b366-80b2eb3d81a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:15.775045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cc900-454a-4cb2-b366-80b2eb3d81a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:15.775708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cc900-454a-4cb2-b366-80b2eb3d81a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:15.776669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cc900-454a-4cb2-b366-80b2eb3d81a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:15.781333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cc900-454a-4cb2-b366-80b2eb3d81a5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:15.790286 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e1b74-f482-46e6-aeb9-05276366865d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:15.792311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e1b74-f482-46e6-aeb9-05276366865d service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:15.845808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 868/1730] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:15.879302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 863/1731] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:15 2022] POST /identity/v3/projects => generated 405 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:15.883433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e1b74-f482-46e6-aeb9-05276366865d service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:15.902733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e1b74-f482-46e6-aeb9-05276366865d service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:16.099952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b20a4-7b68-428e-a785-14cec6855d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:16.105416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b20a4-7b68-428e-a785-14cec6855d8a admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:16.105879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b20a4-7b68-428e-a785-14cec6855d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:16.106260 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b20a4-7b68-428e-a785-14cec6855d8a admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:16.107121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b20a4-7b68-428e-a785-14cec6855d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:16.111000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b20a4-7b68-428e-a785-14cec6855d8a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:16.180226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 869/1732] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:15 2022] POST /identity/v3/users => generated 309 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:16.184391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cc900-454a-4cb2-b366-80b2eb3d81a5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:16.198804 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cc900-454a-4cb2-b366-80b2eb3d81a5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:16.216234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1a3b8-5007-4db6-8796-145113349d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:16.217281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1a3b8-5007-4db6-8796-145113349d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:16.219863 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1a3b8-5007-4db6-8796-145113349d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:16.220456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1a3b8-5007-4db6-8796-145113349d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:16.221704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1a3b8-5007-4db6-8796-145113349d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:16.230565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1a3b8-5007-4db6-8796-145113349d12 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:16.258886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 864/1733] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:16.263025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b20a4-7b68-428e-a785-14cec6855d8a admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:16.282219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b20a4-7b68-428e-a785-14cec6855d8a admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:16.292718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef45a7-9602-45db-8a7d-f281f50e667e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:16.293811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef45a7-9602-45db-8a7d-f281f50e667e admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:16.294265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef45a7-9602-45db-8a7d-f281f50e667e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:16.294645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef45a7-9602-45db-8a7d-f281f50e667e admin admin] PATH_INFO: `/v3/projects/fb1efea40c7541d8893435d4426cdc4d/users/414fdb4f291142c9b738bec75988b582/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:16.337225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef45a7-9602-45db-8a7d-f281f50e667e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1efea40c7541d8893435d4426cdc4d, user_id=414fdb4f291142c9b738bec75988b582, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:16.347744 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef45a7-9602-45db-8a7d-f281f50e667e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:16.390247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 870/1734] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:16 2022] PUT /identity/v3/projects/fb1efea40c7541d8893435d4426cdc4d/users/414fdb4f291142c9b738bec75988b582/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:16.394137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1a3b8-5007-4db6-8796-145113349d12 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:16.411456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1a3b8-5007-4db6-8796-145113349d12 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:16.423610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:16.602758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb732a8-838c-4540-a51a-00df249ac46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:16.603667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb732a8-838c-4540-a51a-00df249ac46a admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:16.605541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb732a8-838c-4540-a51a-00df249ac46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:16.606855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb732a8-838c-4540-a51a-00df249ac46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:16.608463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb732a8-838c-4540-a51a-00df249ac46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:16.613992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb732a8-838c-4540-a51a-00df249ac46a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:16.653293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef45a7-9602-45db-8a7d-f281f50e667e admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:16.654188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 865/1735] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:16.685357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef45a7-9602-45db-8a7d-f281f50e667e admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:16.716123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6289-5169-4408-8e85-302b4a62b511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:16.717230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6289-5169-4408-8e85-302b4a62b511 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:16.717510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6289-5169-4408-8e85-302b4a62b511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:16.718016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6289-5169-4408-8e85-302b4a62b511 admin admin] PATH_INFO: `/v3/projects/fb1efea40c7541d8893435d4426cdc4d/users/414fdb4f291142c9b738bec75988b582/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:16.726915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d6289-5169-4408-8e85-302b4a62b511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1efea40c7541d8893435d4426cdc4d, user_id=414fdb4f291142c9b738bec75988b582, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:16.731510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d6289-5169-4408-8e85-302b4a62b511 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:16.767981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 871/1736] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:16 2022] PUT /identity/v3/projects/fb1efea40c7541d8893435d4426cdc4d/users/414fdb4f291142c9b738bec75988b582/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:54:16.934229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84eab9-28c8-40e3-b199-e92778b0e422 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:16.934229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84eab9-28c8-40e3-b199-e92778b0e422 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:16.934229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84eab9-28c8-40e3-b199-e92778b0e422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:16.999297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-8e84eab9-28c8-40e3-b199-e92778b0e422 None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:17.001869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8e84eab9-28c8-40e3-b199-e92778b0e422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:17.219473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 866/1737] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 05:54:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:17.595847 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6289-5169-4408-8e85-302b4a62b511 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:17.599582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-234d6289-5169-4408-8e85-302b4a62b511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:17.615438 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:17.864556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6289-5169-4408-8e85-302b4a62b511 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:17.880030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47d6e3-757b-4728-93e3-4692497af5b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:17.881169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d6e3-757b-4728-93e3-4692497af5b3 service swift] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:17.881598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d6e3-757b-4728-93e3-4692497af5b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:17.882020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d6e3-757b-4728-93e3-4692497af5b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:17.898148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47d6e3-757b-4728-93e3-4692497af5b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:17.901494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47d6e3-757b-4728-93e3-4692497af5b3 service swift] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:17.923648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 872/1738] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 05:54:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 05:54:19.089744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84eab9-28c8-40e3-b199-e92778b0e422 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:19.091441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8e84eab9-28c8-40e3-b199-e92778b0e422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:19.353559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84eab9-28c8-40e3-b199-e92778b0e422 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:19.368483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:19.380308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:19.381221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:19.381714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] PATH_INFO: `/v3/users/da26676819d4404fac31efeecf89d913` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:19.385379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=da26676819d4404fac31efeecf89d913)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:19.393349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:19.594865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da26676819d4404fac31efeecf89d913'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:19.609979 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:19.617633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da26676819d4404fac31efeecf89d913'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:19.647368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da26676819d4404fac31efeecf89d913'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:19.650008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da26676819d4404fac31efeecf89d913'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:19.652399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da26676819d4404fac31efeecf89d913'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:19.712273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 867/1739] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:19 2022] DELETE /identity/v3/users/da26676819d4404fac31efeecf89d913 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:19.717083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941845c-936c-4cc7-880e-bff352bcb805 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:19.717722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941845c-936c-4cc7-880e-bff352bcb805 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:19.717814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941845c-936c-4cc7-880e-bff352bcb805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:19.776463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-4941845c-936c-4cc7-880e-bff352bcb805 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:19.777997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4941845c-936c-4cc7-880e-bff352bcb805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:19.990171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 873/1740] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:20.082124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:20.084122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:20.209445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77887f2a-ed78-4c1c-8e23-5aabca9a34bf admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:20.223997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27735dc-a168-4b4b-a4ac-d76b9b5f89ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:20.224690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27735dc-a168-4b4b-a4ac-d76b9b5f89ae service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:20.224966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27735dc-a168-4b4b-a4ac-d76b9b5f89ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:20.225247 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27735dc-a168-4b4b-a4ac-d76b9b5f89ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:20.244609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27735dc-a168-4b4b-a4ac-d76b9b5f89ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:20.246502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27735dc-a168-4b4b-a4ac-d76b9b5f89ae service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:20.276874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 868/1741] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:20.803800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941845c-936c-4cc7-880e-bff352bcb805 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:20.805876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4941845c-936c-4cc7-880e-bff352bcb805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:20.821815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941845c-936c-4cc7-880e-bff352bcb805 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:20.828633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:20.829848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:20.830300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:20.830575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] PATH_INFO: `/v3/projects/3a72d2feb4844d9cba103aa34d7352ab` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:20.832095 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a72d2feb4844d9cba103aa34d7352ab)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:20.834967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:20.930964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a72d2feb4844d9cba103aa34d7352ab', 'request_id': 'req-80aece76-93e0-4534-bc68-0ee41da50d7a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:20.938738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a72d2feb4844d9cba103aa34d7352ab', 'request_id': 'req-80aece76-93e0-4534-bc68-0ee41da50d7a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:20.946870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a72d2feb4844d9cba103aa34d7352ab', 'request_id': 'req-80aece76-93e0-4534-bc68-0ee41da50d7a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:20.970043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] The token cache is being invalidate because project 3a72d2feb4844d9cba103aa34d7352ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:20.971110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:20.971439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-80aece76-93e0-4534-bc68-0ee41da50d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:20.973823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 874/1742] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:20 2022] DELETE /identity/v3/projects/3a72d2feb4844d9cba103aa34d7352ab => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:20.978868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27735dc-a168-4b4b-a4ac-d76b9b5f89ae service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:20.991434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27735dc-a168-4b4b-a4ac-d76b9b5f89ae service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:21.104296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:21.105466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:21.106492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:21.107646 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] PATH_INFO: `/v3/users/6ed165c990d24da2983e5d08960f841b` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:21.109790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ed165c990d24da2983e5d08960f841b)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:21.112789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:21.174851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5dfdb-a174-4d86-b7a1-c1c2d300d822 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:21.175484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5dfdb-a174-4d86-b7a1-c1c2d300d822 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:21.176338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5dfdb-a174-4d86-b7a1-c1c2d300d822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:21.197309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-53c5dfdb-a174-4d86-b7a1-c1c2d300d822 None None] MFA Rules not processed for user `32703b5451cf48b09966a7412b747099`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:21.198591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-53c5dfdb-a174-4d86-b7a1-c1c2d300d822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:21.203813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ed165c990d24da2983e5d08960f841b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:21.212749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ed165c990d24da2983e5d08960f841b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:21.215476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:21.228495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ed165c990d24da2983e5d08960f841b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:21.229551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ed165c990d24da2983e5d08960f841b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:21.230401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44cc4d8f-acdc-46ef-9329-eb6e8d49b2a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ed165c990d24da2983e5d08960f841b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:21.244248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:21.259995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 869/1743] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:20 2022] DELETE /identity/v3/users/6ed165c990d24da2983e5d08960f841b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:21.300850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 875/1744] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:21.403648 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571b26d-ae64-45d7-b58f-281b7c41bcf4 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:21.403925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571b26d-ae64-45d7-b58f-281b7c41bcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:21.404204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571b26d-ae64-45d7-b58f-281b7c41bcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:21.454313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2571b26d-ae64-45d7-b58f-281b7c41bcf4 None None] MFA Rules not processed for user `414fdb4f291142c9b738bec75988b582`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:21.455587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2571b26d-ae64-45d7-b58f-281b7c41bcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:21.634539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 870/1745] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:21 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:21.949234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01853b03-1a09-4922-9184-32f3429893ad None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:21.949696 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01853b03-1a09-4922-9184-32f3429893ad None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:21.949782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01853b03-1a09-4922-9184-32f3429893ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:22.000441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-01853b03-1a09-4922-9184-32f3429893ad None None] MFA Rules not processed for user `e367c76dfa7f4d2791e60ce148f352c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:22.004912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-01853b03-1a09-4922-9184-32f3429893ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:22.080296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571b26d-ae64-45d7-b58f-281b7c41bcf4 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:22.082087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2571b26d-ae64-45d7-b58f-281b7c41bcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:22.148386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 876/1746] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:22.240318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571b26d-ae64-45d7-b58f-281b7c41bcf4 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:22.249723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:22.250982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:22.251545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:22.252027 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] PATH_INFO: `/v3/projects/481ba4ca7a10481d988c7434a321c2e0` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:22.264924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=481ba4ca7a10481d988c7434a321c2e0)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:22.266901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:22.354305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '481ba4ca7a10481d988c7434a321c2e0', 'request_id': 'req-3beb0f3a-4395-4f30-a52c-ea4177d9db65'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:22.361460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '481ba4ca7a10481d988c7434a321c2e0', 'request_id': 'req-3beb0f3a-4395-4f30-a52c-ea4177d9db65'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:22.370854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '481ba4ca7a10481d988c7434a321c2e0', 'request_id': 'req-3beb0f3a-4395-4f30-a52c-ea4177d9db65'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:22.392154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] The token cache is being invalidate because project 481ba4ca7a10481d988c7434a321c2e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:22.392853 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:22.393915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-3beb0f3a-4395-4f30-a52c-ea4177d9db65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:22.397175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 871/1747] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:22 2022] DELETE /identity/v3/projects/481ba4ca7a10481d988c7434a321c2e0 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:22.402778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be7891-25ee-4fa3-93c7-b3f4630a5b0e None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:22.403153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be7891-25ee-4fa3-93c7-b3f4630a5b0e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:22.403679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be7891-25ee-4fa3-93c7-b3f4630a5b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:22.449174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b6be7891-25ee-4fa3-93c7-b3f4630a5b0e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:22.450556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b6be7891-25ee-4fa3-93c7-b3f4630a5b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:22.551858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 877/1748] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:22.553246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d58dfd-25f5-4303-8856-a8daf3efbae8 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:22.553522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d58dfd-25f5-4303-8856-a8daf3efbae8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:22.553819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d58dfd-25f5-4303-8856-a8daf3efbae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:22.584839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:22.646713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-52d58dfd-25f5-4303-8856-a8daf3efbae8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:22.653646 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-52d58dfd-25f5-4303-8856-a8daf3efbae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:22.684909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 872/1749] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:22.690116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be7891-25ee-4fa3-93c7-b3f4630a5b0e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:22.734180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be7891-25ee-4fa3-93c7-b3f4630a5b0e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:22.760220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d2085-5a17-437b-8d0b-b31a339f0a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:22.761283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d2085-5a17-437b-8d0b-b31a339f0a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:22.761571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d2085-5a17-437b-8d0b-b31a339f0a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:22.761855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d2085-5a17-437b-8d0b-b31a339f0a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:22.776760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d2085-5a17-437b-8d0b-b31a339f0a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:22.777090 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d2085-5a17-437b-8d0b-b31a339f0a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:22.785236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d2085-5a17-437b-8d0b-b31a339f0a28 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:22.802248 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 878/1750] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:22.808277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d58dfd-25f5-4303-8856-a8daf3efbae8 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:22.819100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d58dfd-25f5-4303-8856-a8daf3efbae8 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:22.830333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b8e30-d941-4f1c-bac1-2f616da39e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:22.831946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b8e30-d941-4f1c-bac1-2f616da39e05 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:22.832689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b8e30-d941-4f1c-bac1-2f616da39e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:22.834887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b8e30-d941-4f1c-bac1-2f616da39e05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:22.835861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b8e30-d941-4f1c-bac1-2f616da39e05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:22.840128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b8e30-d941-4f1c-bac1-2f616da39e05 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:22.956432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 873/1751] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:22 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:22.960435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d2085-5a17-437b-8d0b-b31a339f0a28 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:22.974651 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183aaec0-01d0-43c5-ba1b-978ee6954fed None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:22.975133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183aaec0-01d0-43c5-ba1b-978ee6954fed None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:22.975432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183aaec0-01d0-43c5-ba1b-978ee6954fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:22.987631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d2085-5a17-437b-8d0b-b31a339f0a28 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:23.016757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-183aaec0-01d0-43c5-ba1b-978ee6954fed None None] MFA Rules not processed for user `817b1a026f4b443daea514338ad8531a`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:23.022702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-183aaec0-01d0-43c5-ba1b-978ee6954fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:23.239978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:23.246286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 874/1752] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Jul 4 05:54:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:23.262586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618395e-7ec8-47d0-a4e7-dcea65ff322e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:23.264444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618395e-7ec8-47d0-a4e7-dcea65ff322e admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:23.269210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618395e-7ec8-47d0-a4e7-dcea65ff322e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:23.270361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618395e-7ec8-47d0-a4e7-dcea65ff322e admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:23.271461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618395e-7ec8-47d0-a4e7-dcea65ff322e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:23.275092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618395e-7ec8-47d0-a4e7-dcea65ff322e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:23.347960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183aaec0-01d0-43c5-ba1b-978ee6954fed None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:23.350095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-183aaec0-01d0-43c5-ba1b-978ee6954fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:23.357331 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 879/1753] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:22 2022] POST /identity/v3/users => generated 289 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:23.362679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618395e-7ec8-47d0-a4e7-dcea65ff322e admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:23.372466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183aaec0-01d0-43c5-ba1b-978ee6954fed None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:23.381945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41253d6-2bed-4d2e-b6fe-62c47fff3ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:23.383120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41253d6-2bed-4d2e-b6fe-62c47fff3ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:23.383572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41253d6-2bed-4d2e-b6fe-62c47fff3ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:23.384141 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41253d6-2bed-4d2e-b6fe-62c47fff3ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:23.399933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618395e-7ec8-47d0-a4e7-dcea65ff322e admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:23.408661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41253d6-2bed-4d2e-b6fe-62c47fff3ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:23.410202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41253d6-2bed-4d2e-b6fe-62c47fff3ce9 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:23.424091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b303218-286f-4f10-be55-5d8364db0038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:23.428434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b303218-286f-4f10-be55-5d8364db0038 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:23.429277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b303218-286f-4f10-be55-5d8364db0038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:23.429664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b303218-286f-4f10-be55-5d8364db0038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:23.430479 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 875/1754] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:23.436752 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b303218-286f-4f10-be55-5d8364db0038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:23.439056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b303218-286f-4f10-be55-5d8364db0038 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:23.467078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 880/1755] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:23.472257 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41253d6-2bed-4d2e-b6fe-62c47fff3ce9 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:23.507226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41253d6-2bed-4d2e-b6fe-62c47fff3ce9 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:23.522702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a0306-7792-4b42-aec7-e08f33af3fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:23.524019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a0306-7792-4b42-aec7-e08f33af3fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:23.524522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a0306-7792-4b42-aec7-e08f33af3fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:23.524990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a0306-7792-4b42-aec7-e08f33af3fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:23.526840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a0306-7792-4b42-aec7-e08f33af3fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:23.531537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a0306-7792-4b42-aec7-e08f33af3fcb admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:23.568485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 876/1756] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:23.577455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b303218-286f-4f10-be55-5d8364db0038 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:23.588268 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b303218-286f-4f10-be55-5d8364db0038 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:23.614270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b156f1-29db-472b-a24e-7577ef691c86 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:23.615974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b156f1-29db-472b-a24e-7577ef691c86 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:23.616328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b156f1-29db-472b-a24e-7577ef691c86 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:23.616910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63368748-e3bc-4296-aab8-b391cd3b4b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:23.618928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 877/1757] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 05:54:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:54:23.625466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63368748-e3bc-4296-aab8-b391cd3b4b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:23.628607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63368748-e3bc-4296-aab8-b391cd3b4b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:23.629309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63368748-e3bc-4296-aab8-b391cd3b4b6b admin admin] PATH_INFO: `/v3/projects/2f0809822766482da031a81e83c6e53a/users/6a391589707442d9b8a0d14ada7f1b08/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:23.652723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c6290-2873-4c4e-9750-8ee81d8e96b5 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:23.653049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c6290-2873-4c4e-9750-8ee81d8e96b5 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:23.653356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c6290-2873-4c4e-9750-8ee81d8e96b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:23.703946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63368748-e3bc-4296-aab8-b391cd3b4b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0809822766482da031a81e83c6e53a, user_id=6a391589707442d9b8a0d14ada7f1b08, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:23.705306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-508c6290-2873-4c4e-9750-8ee81d8e96b5 None None] MFA Rules not processed for user `486de932d1db422ba2c3d84142f9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:23.706823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63368748-e3bc-4296-aab8-b391cd3b4b6b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:23.709497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-508c6290-2873-4c4e-9750-8ee81d8e96b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:23.791414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 881/1758] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:23 2022] PUT /identity/v3/projects/2f0809822766482da031a81e83c6e53a/users/6a391589707442d9b8a0d14ada7f1b08/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:23.795467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f71cf-b567-4d3e-a98e-094f08a2deae None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:23.795846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f71cf-b567-4d3e-a98e-094f08a2deae None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:23.796545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f71cf-b567-4d3e-a98e-094f08a2deae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:23.886257 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 878/1759] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 05:54:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:23.962088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-776f71cf-b567-4d3e-a98e-094f08a2deae None None] MFA Rules not processed for user `6a391589707442d9b8a0d14ada7f1b08`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:23.963687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-776f71cf-b567-4d3e-a98e-094f08a2deae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:24.210219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 882/1760] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:24.222986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c6290-2873-4c4e-9750-8ee81d8e96b5 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:24.224852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-508c6290-2873-4c4e-9750-8ee81d8e96b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:24.387111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c6290-2873-4c4e-9750-8ee81d8e96b5 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:24.412473 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f71cf-b567-4d3e-a98e-094f08a2deae None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:24.426515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-776f71cf-b567-4d3e-a98e-094f08a2deae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:24.428131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:24.541327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159f19c-b85f-45c7-a308-df894caf3be4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab33473d40094ec681e538cde250ec2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:24.542141 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159f19c-b85f-45c7-a308-df894caf3be4 service glance] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:24.542764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159f19c-b85f-45c7-a308-df894caf3be4 service glance] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:24.543573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159f19c-b85f-45c7-a308-df894caf3be4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:24.591086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159f19c-b85f-45c7-a308-df894caf3be4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:24.592966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159f19c-b85f-45c7-a308-df894caf3be4 service glance] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:24.626344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 879/1761] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:24.631964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f71cf-b567-4d3e-a98e-094f08a2deae None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:24.646296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc54064-6e10-40c7-9a4f-b773e7695723 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:24.647632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc54064-6e10-40c7-9a4f-b773e7695723 service placement] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:24.648847 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc54064-6e10-40c7-9a4f-b773e7695723 service placement] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:24.649229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc54064-6e10-40c7-9a4f-b773e7695723 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:24.667853 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc54064-6e10-40c7-9a4f-b773e7695723 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:24.673894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc54064-6e10-40c7-9a4f-b773e7695723 service placement] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:24.700133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 883/1762] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:25.208340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159f19c-b85f-45c7-a308-df894caf3be4 service glance] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:25.209983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4159f19c-b85f-45c7-a308-df894caf3be4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:25.517690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159f19c-b85f-45c7-a308-df894caf3be4 service glance] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:25.549182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:25.554958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:25.555930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:25.556212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] PATH_INFO: `/v3/users/9779bb6318d046dc8caea86b3efbc31a` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:25.563228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9779bb6318d046dc8caea86b3efbc31a)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:25.571217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:25.740829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9779bb6318d046dc8caea86b3efbc31a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:25.768331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9779bb6318d046dc8caea86b3efbc31a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:25.795101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9779bb6318d046dc8caea86b3efbc31a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:25.796039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9779bb6318d046dc8caea86b3efbc31a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:25.797164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-749bea50-a829-46ba-8e76-0e0ea282616d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9779bb6318d046dc8caea86b3efbc31a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:25.815100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc54064-6e10-40c7-9a4f-b773e7695723 service placement] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:25.817365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cbc54064-6e10-40c7-9a4f-b773e7695723 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:25.884361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc54064-6e10-40c7-9a4f-b773e7695723 service placement] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:25.894508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 880/1763] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:25 2022] DELETE /identity/v3/users/9779bb6318d046dc8caea86b3efbc31a => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:25.899629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62da96c-064e-4ba8-8a7f-0bcab8ec39eb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:25.899932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62da96c-064e-4ba8-8a7f-0bcab8ec39eb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:25.900208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62da96c-064e-4ba8-8a7f-0bcab8ec39eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:25.900519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:25.910137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:25.912623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:25.912953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] PATH_INFO: `/v3/users/ebcc99ee16da4f928c8372e8d666a307` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:25.917122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebcc99ee16da4f928c8372e8d666a307)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:25.922171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:25.996561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-e62da96c-064e-4ba8-8a7f-0bcab8ec39eb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:25.997809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e62da96c-064e-4ba8-8a7f-0bcab8ec39eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:26.008858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:26.066770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebcc99ee16da4f928c8372e8d666a307'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:26.076345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebcc99ee16da4f928c8372e8d666a307'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:26.077389 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebcc99ee16da4f928c8372e8d666a307'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:26.130077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebcc99ee16da4f928c8372e8d666a307'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:26.172832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebcc99ee16da4f928c8372e8d666a307'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:26.174315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:26.192846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 884/1764] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:25 2022] DELETE /identity/v3/users/ebcc99ee16da4f928c8372e8d666a307 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:26.268698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 881/1765] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:26.356207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:26.358982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:26.506212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c9b1e-f2a3-43b0-9633-09f14f87ec0e admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:26.516842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcacd9-8e33-4fdb-9eed-578ee371413a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:26.516842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcacd9-8e33-4fdb-9eed-578ee371413a service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:26.516842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcacd9-8e33-4fdb-9eed-578ee371413a service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:26.516842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcacd9-8e33-4fdb-9eed-578ee371413a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:26.606095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62da96c-064e-4ba8-8a7f-0bcab8ec39eb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:26.607658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e62da96c-064e-4ba8-8a7f-0bcab8ec39eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:26.631053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62da96c-064e-4ba8-8a7f-0bcab8ec39eb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:26.654772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcacd9-8e33-4fdb-9eed-578ee371413a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:26.659105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcacd9-8e33-4fdb-9eed-578ee371413a service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:26.673170 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 885/1766] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:26.674603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797aa92-9cbe-4fd0-89c3-f1ba0545dfa6 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:26.674896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797aa92-9cbe-4fd0-89c3-f1ba0545dfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:26.675178 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797aa92-9cbe-4fd0-89c3-f1ba0545dfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:26.704854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d797aa92-9cbe-4fd0-89c3-f1ba0545dfa6 None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:26.707283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d797aa92-9cbe-4fd0-89c3-f1ba0545dfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:26.785756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:26.797851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:26.802972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:26.803648 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:26.804776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] PATH_INFO: `/v3/users/4432520048d04008aa5d930828dbae2a` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:26.808433 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4432520048d04008aa5d930828dbae2a)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:26.813066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:26.819176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 886/1767] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 05:54:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:26.838074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797aa92-9cbe-4fd0-89c3-f1ba0545dfa6 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:26.841684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d797aa92-9cbe-4fd0-89c3-f1ba0545dfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:26.964072 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4432520048d04008aa5d930828dbae2a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:26.987827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4432520048d04008aa5d930828dbae2a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:27.009142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797aa92-9cbe-4fd0-89c3-f1ba0545dfa6 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:27.013122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4432520048d04008aa5d930828dbae2a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:27.013691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4432520048d04008aa5d930828dbae2a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:27.014591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1473664d-088d-4eeb-a6c1-ba8e3c46c751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4432520048d04008aa5d930828dbae2a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:27.023507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b872aa-c5e7-4387-9800-d37eb0211c17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:27.025296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b872aa-c5e7-4387-9800-d37eb0211c17 service swift] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:27.025593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b872aa-c5e7-4387-9800-d37eb0211c17 service swift] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:27.026093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b872aa-c5e7-4387-9800-d37eb0211c17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:27.043792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-05b872aa-c5e7-4387-9800-d37eb0211c17 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:27.087161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b872aa-c5e7-4387-9800-d37eb0211c17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:27.089334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b872aa-c5e7-4387-9800-d37eb0211c17 service swift] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:27.089467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 882/1768] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:26 2022] DELETE /identity/v3/users/4432520048d04008aa5d930828dbae2a => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:27.093748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2284e-132c-4e0a-907f-7bcf6ccf95d8 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:27.094041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2284e-132c-4e0a-907f-7bcf6ccf95d8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:27.098628 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2284e-132c-4e0a-907f-7bcf6ccf95d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:27.116179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 887/1769] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 05:54:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 05:54:27.148582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-0df2284e-132c-4e0a-907f-7bcf6ccf95d8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:27.150028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0df2284e-132c-4e0a-907f-7bcf6ccf95d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:27.256557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:27.286069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 883/1770] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:27.330698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b872aa-c5e7-4387-9800-d37eb0211c17 service swift] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:27.331947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-05b872aa-c5e7-4387-9800-d37eb0211c17 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:27.365027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b872aa-c5e7-4387-9800-d37eb0211c17 service swift] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:27.587608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2284e-132c-4e0a-907f-7bcf6ccf95d8 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:27.589581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0df2284e-132c-4e0a-907f-7bcf6ccf95d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:27.603427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ba707-5aff-4829-9251-14daeb6cd131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:27.603954 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2284e-132c-4e0a-907f-7bcf6ccf95d8 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:27.606753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba707-5aff-4829-9251-14daeb6cd131 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:27.607025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba707-5aff-4829-9251-14daeb6cd131 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:27.607363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba707-5aff-4829-9251-14daeb6cd131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:27.610570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:27.613191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:27.613840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:27.614314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] PATH_INFO: `/v3/projects/8a928afbe8c84761b9e7f351cee6bdd5` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:27.616204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a928afbe8c84761b9e7f351cee6bdd5)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:27.618899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:27.658793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ba707-5aff-4829-9251-14daeb6cd131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:27.668372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ba707-5aff-4829-9251-14daeb6cd131 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:27.714181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a928afbe8c84761b9e7f351cee6bdd5', 'request_id': 'req-e2b97b89-e0e7-4277-846c-c6407daeb3fd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:27.731454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a928afbe8c84761b9e7f351cee6bdd5', 'request_id': 'req-e2b97b89-e0e7-4277-846c-c6407daeb3fd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:27.742824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 888/1771] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:27.745703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a928afbe8c84761b9e7f351cee6bdd5', 'request_id': 'req-e2b97b89-e0e7-4277-846c-c6407daeb3fd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:27.797283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] The token cache is being invalidate because project 8a928afbe8c84761b9e7f351cee6bdd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:27.797686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:27.807969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e2b97b89-e0e7-4277-846c-c6407daeb3fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:27.812857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 884/1772] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:27 2022] DELETE /identity/v3/projects/8a928afbe8c84761b9e7f351cee6bdd5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:27.813948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ba707-5aff-4829-9251-14daeb6cd131 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:27.827850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ba707-5aff-4829-9251-14daeb6cd131 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:27.962227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a6a3a-a87e-4b75-bcce-cde28d16d4e5 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:27.962525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a6a3a-a87e-4b75-bcce-cde28d16d4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:27.962809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a6a3a-a87e-4b75-bcce-cde28d16d4e5 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:27.965247 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 885/1773] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 05:54:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:54:28.003914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:28.005300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:28.005803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:28.006546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] PATH_INFO: `/v3/users/414fdb4f291142c9b738bec75988b582` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:28.008217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=414fdb4f291142c9b738bec75988b582)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:28.012943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:28.027759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ac3da-3780-4f10-abb0-115b4fbf5aef None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:28.028045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ac3da-3780-4f10-abb0-115b4fbf5aef None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:28.028327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ac3da-3780-4f10-abb0-115b4fbf5aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:28.091402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:28.095089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-970ac3da-3780-4f10-abb0-115b4fbf5aef None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:28.112180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-970ac3da-3780-4f10-abb0-115b4fbf5aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:28.137047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414fdb4f291142c9b738bec75988b582'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:28.166120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414fdb4f291142c9b738bec75988b582'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:28.167703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414fdb4f291142c9b738bec75988b582'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:28.197266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414fdb4f291142c9b738bec75988b582'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:28.240465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414fdb4f291142c9b738bec75988b582'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:28.263780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 889/1774] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:27 2022] DELETE /identity/v3/users/414fdb4f291142c9b738bec75988b582 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:28.382272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 886/1775] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 05:54:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:29.200747 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:29.202417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:29.353973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c1b6a-79b6-44dd-b687-f5816871e805 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:29.366678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcc41f-947c-4cc6-8f0a-a89415d44250 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:29.367568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc41f-947c-4cc6-8f0a-a89415d44250 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:29.367858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc41f-947c-4cc6-8f0a-a89415d44250 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:29.368140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc41f-947c-4cc6-8f0a-a89415d44250 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:29.412115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcc41f-947c-4cc6-8f0a-a89415d44250 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:29.415749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcc41f-947c-4cc6-8f0a-a89415d44250 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:29.454558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 890/1776] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:29.695764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ac3da-3780-4f10-abb0-115b4fbf5aef None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:29.697365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-970ac3da-3780-4f10-abb0-115b4fbf5aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:29.721760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ac3da-3780-4f10-abb0-115b4fbf5aef None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:29.797806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcc41f-947c-4cc6-8f0a-a89415d44250 service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:29.799219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-eabcc41f-947c-4cc6-8f0a-a89415d44250 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:29.914503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:29.915454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:29.915742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:29.916067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] PATH_INFO: `/v3/projects/512f151955cb43a1bae07da95b4e52f2` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:29.917243 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=512f151955cb43a1bae07da95b4e52f2)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:29.919177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:29.935135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcc41f-947c-4cc6-8f0a-a89415d44250 service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:29.947446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:29.949077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:29.950049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:29.950333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:29.950608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] PATH_INFO: `/v3/projects/fb1efea40c7541d8893435d4426cdc4d` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:29.953785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1efea40c7541d8893435d4426cdc4d)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:29.956477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:30.039430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:30.040570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512f151955cb43a1bae07da95b4e52f2', 'request_id': 'req-60e54b64-781f-45a0-9517-2dff00841b34'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.063530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512f151955cb43a1bae07da95b4e52f2', 'request_id': 'req-60e54b64-781f-45a0-9517-2dff00841b34'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.075704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1efea40c7541d8893435d4426cdc4d', 'request_id': 'req-989e86c1-a7de-41f8-a2af-85d20573a40a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.081561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512f151955cb43a1bae07da95b4e52f2', 'request_id': 'req-60e54b64-781f-45a0-9517-2dff00841b34'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.101040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1efea40c7541d8893435d4426cdc4d', 'request_id': 'req-989e86c1-a7de-41f8-a2af-85d20573a40a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.110446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1efea40c7541d8893435d4426cdc4d', 'request_id': 'req-989e86c1-a7de-41f8-a2af-85d20573a40a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.125230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] The token cache is being invalidate because project 512f151955cb43a1bae07da95b4e52f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:30.127825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.128614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.133150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 887/1777] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:29 2022] DELETE /identity/v3/projects/512f151955cb43a1bae07da95b4e52f2 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:30.134304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:30.136705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] The token cache is being invalidate because project fb1efea40c7541d8893435d4426cdc4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:30.137168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.138056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.138623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:30.143494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 891/1778] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:29 2022] DELETE /identity/v3/projects/fb1efea40c7541d8893435d4426cdc4d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:30.143494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:30.143787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:30.261377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e54b64-781f-45a0-9517-2dff00841b34 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:30.297184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e86c1-a7de-41f8-a2af-85d20573a40a admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:30.311397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:30.313085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:30.313812 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:30.314278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] PATH_INFO: `/v3/users/e7d6e291cd914d5a826118cc7d3cd484` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:30.317719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7d6e291cd914d5a826118cc7d3cd484)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:30.341585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:30.380767 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:30.381788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:30.382118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:30.382442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] PATH_INFO: `/v3/projects/0918524e02a74f1cbcc89142d4d8d012` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:30.387198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0918524e02a74f1cbcc89142d4d8d012)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:30.392995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:30.508000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7d6e291cd914d5a826118cc7d3cd484'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.538235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7d6e291cd914d5a826118cc7d3cd484'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.544225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7d6e291cd914d5a826118cc7d3cd484'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.546239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0918524e02a74f1cbcc89142d4d8d012', 'request_id': 'req-694ff4e0-e08e-4426-9f71-66b5b1403414'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.558809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:30.563004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0918524e02a74f1cbcc89142d4d8d012', 'request_id': 'req-694ff4e0-e08e-4426-9f71-66b5b1403414'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.574557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0918524e02a74f1cbcc89142d4d8d012', 'request_id': 'req-694ff4e0-e08e-4426-9f71-66b5b1403414'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.587185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7d6e291cd914d5a826118cc7d3cd484'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.610079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] The token cache is being invalidate because project 0918524e02a74f1cbcc89142d4d8d012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:30.610404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.615917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-694ff4e0-e08e-4426-9f71-66b5b1403414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.619312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 888/1779] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:30 2022] DELETE /identity/v3/projects/0918524e02a74f1cbcc89142d4d8d012 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:30.619312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecca62-a669-48b0-8cb4-9b1a2b125a6c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:30.619510 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecca62-a669-48b0-8cb4-9b1a2b125a6c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:30.619605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecca62-a669-48b0-8cb4-9b1a2b125a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:30.652235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-9bdd5295-0185-4c2b-af03-7e3c6a90936b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7d6e291cd914d5a826118cc7d3cd484'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:30.667361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-b6ecca62-a669-48b0-8cb4-9b1a2b125a6c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:30.669287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b6ecca62-a669-48b0-8cb4-9b1a2b125a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:30.703282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 892/1780] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:30 2022] DELETE /identity/v3/users/e7d6e291cd914d5a826118cc7d3cd484 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:30.703282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2ade3-d8be-45ea-b935-073243208ed3 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:30.703282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2ade3-d8be-45ea-b935-073243208ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:30.703282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2ade3-d8be-45ea-b935-073243208ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:30.770570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-1cf2ade3-d8be-45ea-b935-073243208ed3 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:30.772304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1cf2ade3-d8be-45ea-b935-073243208ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:30.811451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 889/1781] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:30.817100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae839c7-c77b-419d-8675-7d27c5d60689 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:30.817308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae839c7-c77b-419d-8675-7d27c5d60689 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:30.817652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae839c7-c77b-419d-8675-7d27c5d60689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:30.847150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:30.876534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-5ae839c7-c77b-419d-8675-7d27c5d60689 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:30.877519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 893/1782] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:30.879034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5ae839c7-c77b-419d-8675-7d27c5d60689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:30.887418 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18192c9c-e0ec-4f85-837a-ca18a22c9f42 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:30.887418 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18192c9c-e0ec-4f85-837a-ca18a22c9f42 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:30.887830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18192c9c-e0ec-4f85-837a-ca18a22c9f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:30.907589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 890/1783] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:30.912247 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae839c7-c77b-419d-8675-7d27c5d60689 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:30.925761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-18192c9c-e0ec-4f85-837a-ca18a22c9f42 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:30.927578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-18192c9c-e0ec-4f85-837a-ca18a22c9f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:30.929365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae839c7-c77b-419d-8675-7d27c5d60689 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:30.945345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cee9de-1bd7-4d77-8bc0-10e3b0301643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:30.946152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cee9de-1bd7-4d77-8bc0-10e3b0301643 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:30.946514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cee9de-1bd7-4d77-8bc0-10e3b0301643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:30.946791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cee9de-1bd7-4d77-8bc0-10e3b0301643 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:30.947682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cee9de-1bd7-4d77-8bc0-10e3b0301643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:30.951525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cee9de-1bd7-4d77-8bc0-10e3b0301643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:30.951719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 894/1784] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:30.951824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cee9de-1bd7-4d77-8bc0-10e3b0301643 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:30.952862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18192c9c-e0ec-4f85-837a-ca18a22c9f42 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:30.975479 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18192c9c-e0ec-4f85-837a-ca18a22c9f42 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:30.982683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 891/1785] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:30.987212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cee9de-1bd7-4d77-8bc0-10e3b0301643 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:30.988710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bc8a4-7e85-42a1-9543-097b836a8832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:30.989750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc8a4-7e85-42a1-9543-097b836a8832 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:30.989956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc8a4-7e85-42a1-9543-097b836a8832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:30.990226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc8a4-7e85-42a1-9543-097b836a8832 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:30.990964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bc8a4-7e85-42a1-9543-097b836a8832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:30.991289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bc8a4-7e85-42a1-9543-097b836a8832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:30.996338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bc8a4-7e85-42a1-9543-097b836a8832 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.006234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cee9de-1bd7-4d77-8bc0-10e3b0301643 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:31.018886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801492ce-2333-402f-9a19-f497fc348c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:31.019623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801492ce-2333-402f-9a19-f497fc348c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.019929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801492ce-2333-402f-9a19-f497fc348c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.020232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801492ce-2333-402f-9a19-f497fc348c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.020918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801492ce-2333-402f-9a19-f497fc348c17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:31.022881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801492ce-2333-402f-9a19-f497fc348c17 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.029196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 895/1786] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:31.030428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bc8a4-7e85-42a1-9543-097b836a8832 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:31.049519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bc8a4-7e85-42a1-9543-097b836a8832 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:31.065051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76a0eb-7492-403a-a5af-235299670611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:31.065741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a0eb-7492-403a-a5af-235299670611 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.066021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a0eb-7492-403a-a5af-235299670611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.066290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a0eb-7492-403a-a5af-235299670611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.067076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76a0eb-7492-403a-a5af-235299670611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:31.072367 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76a0eb-7492-403a-a5af-235299670611 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.079550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 892/1787] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:30 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:54:31.083767 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801492ce-2333-402f-9a19-f497fc348c17 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:31.136114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 896/1788] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:31 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:31.137641 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76a0eb-7492-403a-a5af-235299670611 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:31.306345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801492ce-2333-402f-9a19-f497fc348c17 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:31.345736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76a0eb-7492-403a-a5af-235299670611 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:31.360823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0f0bf-999e-47b0-a0b8-23af3ec71d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:31.361529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0f0bf-999e-47b0-a0b8-23af3ec71d8a admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.361819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0f0bf-999e-47b0-a0b8-23af3ec71d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.362085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0f0bf-999e-47b0-a0b8-23af3ec71d8a admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.362978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0f0bf-999e-47b0-a0b8-23af3ec71d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:31.366829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0f0bf-999e-47b0-a0b8-23af3ec71d8a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.416733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 897/1789] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:31 2022] POST /identity/v3/users => generated 286 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:54:31.418546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:31.435436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0f0bf-999e-47b0-a0b8-23af3ec71d8a admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:31.448931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba724e-f145-4677-828b-41f5ac8960eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:31.449151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0f0bf-999e-47b0-a0b8-23af3ec71d8a admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:31.449882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba724e-f145-4677-828b-41f5ac8960eb admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.450258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba724e-f145-4677-828b-41f5ac8960eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.450611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba724e-f145-4677-828b-41f5ac8960eb admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.451685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba724e-f145-4677-828b-41f5ac8960eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:31.458500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba724e-f145-4677-828b-41f5ac8960eb admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.461576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a734ec-9a9a-4ed1-99f9-451adac96146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:31.462336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a734ec-9a9a-4ed1-99f9-451adac96146 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.462615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a734ec-9a9a-4ed1-99f9-451adac96146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.462999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a734ec-9a9a-4ed1-99f9-451adac96146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.463807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a734ec-9a9a-4ed1-99f9-451adac96146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:31.467503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a734ec-9a9a-4ed1-99f9-451adac96146 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.491241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 898/1790] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:31.496301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a734ec-9a9a-4ed1-99f9-451adac96146 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:31.514566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a734ec-9a9a-4ed1-99f9-451adac96146 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:31.528944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 893/1791] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:31 2022] POST /identity/v3/users => generated 283 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:31.529196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcdb29-84af-48b6-b4ca-7f21e34df7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:31.529917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcdb29-84af-48b6-b4ca-7f21e34df7bc admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.531159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba724e-f145-4677-828b-41f5ac8960eb admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:31.531511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcdb29-84af-48b6-b4ca-7f21e34df7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.531787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcdb29-84af-48b6-b4ca-7f21e34df7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.532539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcdb29-84af-48b6-b4ca-7f21e34df7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:31.543311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcdb29-84af-48b6-b4ca-7f21e34df7bc admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.551096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba724e-f145-4677-828b-41f5ac8960eb admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:31.575573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945db2d-21e1-491a-84d6-0b0aa8dd2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:31.576622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945db2d-21e1-491a-84d6-0b0aa8dd2d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.576931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945db2d-21e1-491a-84d6-0b0aa8dd2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.577272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945db2d-21e1-491a-84d6-0b0aa8dd2d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.578107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945db2d-21e1-491a-84d6-0b0aa8dd2d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:31.580663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945db2d-21e1-491a-84d6-0b0aa8dd2d54 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.600888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 899/1792] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:31.605210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcdb29-84af-48b6-b4ca-7f21e34df7bc admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:31.618870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 894/1793] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:31.622550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945db2d-21e1-491a-84d6-0b0aa8dd2d54 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:31.638748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945db2d-21e1-491a-84d6-0b0aa8dd2d54 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:31.649262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106ad6a-fe00-40c8-86b8-ed9982878e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:31.650525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106ad6a-fe00-40c8-86b8-ed9982878e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.651014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106ad6a-fe00-40c8-86b8-ed9982878e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.651913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106ad6a-fe00-40c8-86b8-ed9982878e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.652859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106ad6a-fe00-40c8-86b8-ed9982878e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:31.656475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106ad6a-fe00-40c8-86b8-ed9982878e57 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.674172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcdb29-84af-48b6-b4ca-7f21e34df7bc admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:31.682399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 895/1794] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:31.684831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106ad6a-fe00-40c8-86b8-ed9982878e57 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:31.691000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08133df2-3daa-49ae-8d9d-27dd10ef52a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:31.694997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08133df2-3daa-49ae-8d9d-27dd10ef52a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.695275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08133df2-3daa-49ae-8d9d-27dd10ef52a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.695554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08133df2-3daa-49ae-8d9d-27dd10ef52a3 admin admin] PATH_INFO: `/v3/projects/7705e68009b542cc97318cd75880ecd6/users/3e7874ec8ade4efcb67006400ac916d5/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.700424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106ad6a-fe00-40c8-86b8-ed9982878e57 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:31.706225 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b0c35-e611-4797-beab-bd8e5bbe171d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:31.707153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b0c35-e611-4797-beab-bd8e5bbe171d admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.707568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b0c35-e611-4797-beab-bd8e5bbe171d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.707843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b0c35-e611-4797-beab-bd8e5bbe171d admin admin] PATH_INFO: `/v3/projects/7b4c5894a19c46c9a4150fc26b0f8ced/users/e8c14b75059d4e3c8815d034067e84a0/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.724358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08133df2-3daa-49ae-8d9d-27dd10ef52a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7705e68009b542cc97318cd75880ecd6, user_id=3e7874ec8ade4efcb67006400ac916d5, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:31.728621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08133df2-3daa-49ae-8d9d-27dd10ef52a3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.728813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b0c35-e611-4797-beab-bd8e5bbe171d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4c5894a19c46c9a4150fc26b0f8ced, user_id=e8c14b75059d4e3c8815d034067e84a0, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:31.732568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b0c35-e611-4797-beab-bd8e5bbe171d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:31.749638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 900/1795] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:31 2022] PUT /identity/v3/projects/7705e68009b542cc97318cd75880ecd6/users/3e7874ec8ade4efcb67006400ac916d5/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:31.761662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497b308-22eb-41d4-a634-10b593c9242e None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.762123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497b308-22eb-41d4-a634-10b593c9242e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.762497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497b308-22eb-41d4-a634-10b593c9242e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.781523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 896/1796] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:31 2022] PUT /identity/v3/projects/7b4c5894a19c46c9a4150fc26b0f8ced/users/e8c14b75059d4e3c8815d034067e84a0/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:31.781523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f151e-56bb-447d-aceb-631335845380 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:31.781523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f151e-56bb-447d-aceb-631335845380 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:31.781523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f151e-56bb-447d-aceb-631335845380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:31.806364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-0497b308-22eb-41d4-a634-10b593c9242e None None] MFA Rules not processed for user `3e7874ec8ade4efcb67006400ac916d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:31.807944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0497b308-22eb-41d4-a634-10b593c9242e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:31.848247 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-911f151e-56bb-447d-aceb-631335845380 None None] MFA Rules not processed for user `e8c14b75059d4e3c8815d034067e84a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:31.852425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-911f151e-56bb-447d-aceb-631335845380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:31.863698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:31.993516 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 901/1797] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:32.000869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497b308-22eb-41d4-a634-10b593c9242e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:32.022814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 897/1798] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:32.030951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f151e-56bb-447d-aceb-631335845380 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:32.171483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f151e-56bb-447d-aceb-631335845380 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:32.176386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497b308-22eb-41d4-a634-10b593c9242e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:32.178061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a1118-930e-472a-afe1-1b734d963a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:32.178835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a1118-930e-472a-afe1-1b734d963a07 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:32.179126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a1118-930e-472a-afe1-1b734d963a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:32.179437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a1118-930e-472a-afe1-1b734d963a07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:32.180158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a1118-930e-472a-afe1-1b734d963a07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:32.183006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a1118-930e-472a-afe1-1b734d963a07 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:32.191734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1550254-d8bd-4f1c-b87a-fa0ae27bd996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:32.192983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1550254-d8bd-4f1c-b87a-fa0ae27bd996 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:32.193459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1550254-d8bd-4f1c-b87a-fa0ae27bd996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:32.193970 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1550254-d8bd-4f1c-b87a-fa0ae27bd996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:32.196186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1550254-d8bd-4f1c-b87a-fa0ae27bd996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:32.198387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1550254-d8bd-4f1c-b87a-fa0ae27bd996 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:32.235200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 898/1799] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:32 2022] POST /identity/v3/projects => generated 361 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:32.237136 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a1118-930e-472a-afe1-1b734d963a07 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:32.247742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:32.279485 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 902/1800] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:31 2022] POST /identity/v3/projects => generated 371 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:32.284082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1550254-d8bd-4f1c-b87a-fa0ae27bd996 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:32.368029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a1118-930e-472a-afe1-1b734d963a07 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:32.440554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1550254-d8bd-4f1c-b87a-fa0ae27bd996 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:32.452656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41439554-7f3b-48d7-97c0-acf18e703c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:32.455525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41439554-7f3b-48d7-97c0-acf18e703c08 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:32.456203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41439554-7f3b-48d7-97c0-acf18e703c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:32.458302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41439554-7f3b-48d7-97c0-acf18e703c08 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:32.462449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41439554-7f3b-48d7-97c0-acf18e703c08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:32.471821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41439554-7f3b-48d7-97c0-acf18e703c08 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:32.528160 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29505222-e22d-4a28-992a-13f54b790438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:32.528865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29505222-e22d-4a28-992a-13f54b790438 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:32.529145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29505222-e22d-4a28-992a-13f54b790438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:32.529416 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29505222-e22d-4a28-992a-13f54b790438 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:32.530571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29505222-e22d-4a28-992a-13f54b790438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:32.534092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 903/1801] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:32 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:32.534587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29505222-e22d-4a28-992a-13f54b790438 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:32.535299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41439554-7f3b-48d7-97c0-acf18e703c08 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:32.547541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41439554-7f3b-48d7-97c0-acf18e703c08 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:32.553421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ff5f5-17b5-4125-995b-c683d4a04f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:32.554149 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ff5f5-17b5-4125-995b-c683d4a04f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:32.554438 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ff5f5-17b5-4125-995b-c683d4a04f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:32.554718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ff5f5-17b5-4125-995b-c683d4a04f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:32.555501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ff5f5-17b5-4125-995b-c683d4a04f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:32.557269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ff5f5-17b5-4125-995b-c683d4a04f6c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:32.573240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 904/1802] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:32.574183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ff5f5-17b5-4125-995b-c683d4a04f6c admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:32.579219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 899/1803] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:32 2022] POST /identity/v3/users => generated 287 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:32.584325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29505222-e22d-4a28-992a-13f54b790438 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:32.585883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ff5f5-17b5-4125-995b-c683d4a04f6c admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:32.591028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c0462-77f1-445a-a417-6eaf75e3cdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:32.592080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c0462-77f1-445a-a417-6eaf75e3cdd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:32.592449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c0462-77f1-445a-a417-6eaf75e3cdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:32.592660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c0462-77f1-445a-a417-6eaf75e3cdd9 admin admin] PATH_INFO: `/v3/projects/b9fcbf7d7fa94b679eae037650dba255/users/a737b075e35b43d5ac93aea960ab2664/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:32.613868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29505222-e22d-4a28-992a-13f54b790438 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:32.615538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c0462-77f1-445a-a417-6eaf75e3cdd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fcbf7d7fa94b679eae037650dba255, user_id=a737b075e35b43d5ac93aea960ab2664, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:32.623457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c0462-77f1-445a-a417-6eaf75e3cdd9 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:32.629229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c8e6d-c300-487e-9809-da20595b3141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:32.630374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c8e6d-c300-487e-9809-da20595b3141 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:32.630759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c8e6d-c300-487e-9809-da20595b3141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:32.631029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c8e6d-c300-487e-9809-da20595b3141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:32.631914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c8e6d-c300-487e-9809-da20595b3141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:32.634127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c8e6d-c300-487e-9809-da20595b3141 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:32.681759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 905/1805] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:32 2022] PUT /identity/v3/projects/b9fcbf7d7fa94b679eae037650dba255/users/a737b075e35b43d5ac93aea960ab2664/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:32.682259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 900/1805] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:32.683573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c0462-77f1-445a-a417-6eaf75e3cdd9 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:32.686882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c8e6d-c300-487e-9809-da20595b3141 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:32.702670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:32.720829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c8e6d-c300-487e-9809-da20595b3141 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:32.895030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c0462-77f1-445a-a417-6eaf75e3cdd9 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:32.912295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c5f9f-cc62-405b-8788-56cb48f307d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:32.913227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c5f9f-cc62-405b-8788-56cb48f307d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:32.913945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c5f9f-cc62-405b-8788-56cb48f307d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:32.914783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c5f9f-cc62-405b-8788-56cb48f307d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:32.917913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c5f9f-cc62-405b-8788-56cb48f307d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:32.919749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c5f9f-cc62-405b-8788-56cb48f307d1 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:32.943223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 906/1806] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:32.949516 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c5f9f-cc62-405b-8788-56cb48f307d1 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:32.950606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148ba7c-6bb1-45f5-a2e7-5a0957452018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:32.951820 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148ba7c-6bb1-45f5-a2e7-5a0957452018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:32.952242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148ba7c-6bb1-45f5-a2e7-5a0957452018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:32.952608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148ba7c-6bb1-45f5-a2e7-5a0957452018 admin admin] PATH_INFO: `/v3/projects/d833e893e28c4e60831a10a67098b92e/users/0761eeeda9894151bb595cf92a491732/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:32.974286 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c5f9f-cc62-405b-8788-56cb48f307d1 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:32.985598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96254d37-3b36-4e42-b7de-bdd3c232cb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:32.986878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254d37-3b36-4e42-b7de-bdd3c232cb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:32.988063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254d37-3b36-4e42-b7de-bdd3c232cb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:32.989068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254d37-3b36-4e42-b7de-bdd3c232cb03 admin admin] PATH_INFO: `/v3/projects/b9fcbf7d7fa94b679eae037650dba255/users/a737b075e35b43d5ac93aea960ab2664/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:32.990176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148ba7c-6bb1-45f5-a2e7-5a0957452018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d833e893e28c4e60831a10a67098b92e, user_id=0761eeeda9894151bb595cf92a491732, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:32.995875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96254d37-3b36-4e42-b7de-bdd3c232cb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fcbf7d7fa94b679eae037650dba255, user_id=a737b075e35b43d5ac93aea960ab2664, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:33.000647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96254d37-3b36-4e42-b7de-bdd3c232cb03 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:33.001141 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148ba7c-6bb1-45f5-a2e7-5a0957452018 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:33.030191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 901/1807] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:32 2022] PUT /identity/v3/projects/d833e893e28c4e60831a10a67098b92e/users/0761eeeda9894151bb595cf92a491732/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:33.043831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148ba7c-6bb1-45f5-a2e7-5a0957452018 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:33.044745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 907/1808] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:32 2022] PUT /identity/v3/projects/b9fcbf7d7fa94b679eae037650dba255/users/a737b075e35b43d5ac93aea960ab2664/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:33.048188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbe18e-6824-459c-933a-2a6021630be0 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:33.050767 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbe18e-6824-459c-933a-2a6021630be0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:33.051295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbe18e-6824-459c-933a-2a6021630be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:33.092970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:33.152683 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-3abbe18e-6824-459c-933a-2a6021630be0 None None] MFA Rules not processed for user `a737b075e35b43d5ac93aea960ab2664`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:33.157570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3abbe18e-6824-459c-933a-2a6021630be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:33.275394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148ba7c-6bb1-45f5-a2e7-5a0957452018 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:33.295349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a63ffb-9603-4ddc-a181-8070a95e9a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:33.295833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63ffb-9603-4ddc-a181-8070a95e9a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:33.297455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63ffb-9603-4ddc-a181-8070a95e9a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:33.297455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63ffb-9603-4ddc-a181-8070a95e9a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:33.304048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a63ffb-9603-4ddc-a181-8070a95e9a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:33.305958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a63ffb-9603-4ddc-a181-8070a95e9a0e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:33.349854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 902/1809] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:33.356370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a63ffb-9603-4ddc-a181-8070a95e9a0e admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:33.386797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a63ffb-9603-4ddc-a181-8070a95e9a0e admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:33.415579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0dd0d-7698-448f-9ec8-74c9c2b99f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:33.416771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0dd0d-7698-448f-9ec8-74c9c2b99f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:33.417157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0dd0d-7698-448f-9ec8-74c9c2b99f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:33.417565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0dd0d-7698-448f-9ec8-74c9c2b99f87 admin admin] PATH_INFO: `/v3/projects/d833e893e28c4e60831a10a67098b92e/users/0761eeeda9894151bb595cf92a491732/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:33.421954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 908/1810] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:33 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:33.428287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0dd0d-7698-448f-9ec8-74c9c2b99f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d833e893e28c4e60831a10a67098b92e, user_id=0761eeeda9894151bb595cf92a491732, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:33.436712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbe18e-6824-459c-933a-2a6021630be0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:33.438488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0dd0d-7698-448f-9ec8-74c9c2b99f87 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:33.442014 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3abbe18e-6824-459c-933a-2a6021630be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:33.492115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 903/1811] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:33 2022] PUT /identity/v3/projects/d833e893e28c4e60831a10a67098b92e/users/0761eeeda9894151bb595cf92a491732/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:33.498255 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42234e-1e67-4f54-8de2-9eafb2c69a98 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:33.498649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42234e-1e67-4f54-8de2-9eafb2c69a98 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:33.509222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42234e-1e67-4f54-8de2-9eafb2c69a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:33.592832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-5d42234e-1e67-4f54-8de2-9eafb2c69a98 None None] MFA Rules not processed for user `0761eeeda9894151bb595cf92a491732`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:33.594241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5d42234e-1e67-4f54-8de2-9eafb2c69a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:33.659079 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbe18e-6824-459c-933a-2a6021630be0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:33.739126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:33.763630 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cfd7d-e641-4800-b8a9-6a7baab9a3a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:33.767311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cfd7d-e641-4800-b8a9-6a7baab9a3a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:33.767311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cfd7d-e641-4800-b8a9-6a7baab9a3a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:33.767311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cfd7d-e641-4800-b8a9-6a7baab9a3a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:33.825353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 904/1812] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:33 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:33.894933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42234e-1e67-4f54-8de2-9eafb2c69a98 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:33.897186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5d42234e-1e67-4f54-8de2-9eafb2c69a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:33.920224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cfd7d-e641-4800-b8a9-6a7baab9a3a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:33.922352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42234e-1e67-4f54-8de2-9eafb2c69a98 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:33.926431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cfd7d-e641-4800-b8a9-6a7baab9a3a0 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:33.930627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc76c3f-674c-45e4-8d78-a5a4aa7358dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:33.931759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc76c3f-674c-45e4-8d78-a5a4aa7358dd service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:33.932198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc76c3f-674c-45e4-8d78-a5a4aa7358dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:33.932619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc76c3f-674c-45e4-8d78-a5a4aa7358dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:33.946403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 909/1813] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:33 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:33.962749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc76c3f-674c-45e4-8d78-a5a4aa7358dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:33.970125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc76c3f-674c-45e4-8d78-a5a4aa7358dd service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:34.046093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 905/1814] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:33 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:34.071572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cfd7d-e641-4800-b8a9-6a7baab9a3a0 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:34.102467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cfd7d-e641-4800-b8a9-6a7baab9a3a0 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:34.108157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c94fe7-4017-4e63-a698-69befb55705c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:34.109433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c94fe7-4017-4e63-a698-69befb55705c service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:34.110080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c94fe7-4017-4e63-a698-69befb55705c service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:34.110825 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c94fe7-4017-4e63-a698-69befb55705c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:34.208046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c94fe7-4017-4e63-a698-69befb55705c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:34.210121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c94fe7-4017-4e63-a698-69befb55705c service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:34.229124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc76c3f-674c-45e4-8d78-a5a4aa7358dd service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:34.230668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ebc76c3f-674c-45e4-8d78-a5a4aa7358dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:34.231858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 910/1815] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:34.262033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6700c4-d05f-4a06-be23-3401b40161ff None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:34.262389 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6700c4-d05f-4a06-be23-3401b40161ff None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:34.262651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6700c4-d05f-4a06-be23-3401b40161ff None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:34.264569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 911/1816] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 05:54:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 05:54:34.283482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4c07-997a-447e-8cca-25786eed79b1 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:34.283774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4c07-997a-447e-8cca-25786eed79b1 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:34.284054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4c07-997a-447e-8cca-25786eed79b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:34.342173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d83c4c07-997a-447e-8cca-25786eed79b1 None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:34.343665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d83c4c07-997a-447e-8cca-25786eed79b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:34.480182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc76c3f-674c-45e4-8d78-a5a4aa7358dd service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:34.526681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:34.530389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:34.530681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:34.530963 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] PATH_INFO: `/v3/projects/6a2311b2aa2340c5b5d2eab17913b411` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:34.541188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 912/1817] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 05:54:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:34.545775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a2311b2aa2340c5b5d2eab17913b411)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:34.563105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:34.594038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:34.598943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c4c07-997a-447e-8cca-25786eed79b1 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:34.600727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d83c4c07-997a-447e-8cca-25786eed79b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:34.623054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c4c07-997a-447e-8cca-25786eed79b1 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:34.659770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a2311b2aa2340c5b5d2eab17913b411', 'request_id': 'req-154b301a-f695-47c5-88f6-f0c8db1bd6f8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:34.661301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43c5cf-b6db-4ed6-bb4a-5ccf1604cec7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:34.662424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43c5cf-b6db-4ed6-bb4a-5ccf1604cec7 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:34.662983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43c5cf-b6db-4ed6-bb4a-5ccf1604cec7 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:34.663427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43c5cf-b6db-4ed6-bb4a-5ccf1604cec7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:34.670171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a2311b2aa2340c5b5d2eab17913b411', 'request_id': 'req-154b301a-f695-47c5-88f6-f0c8db1bd6f8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:34.680163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a2311b2aa2340c5b5d2eab17913b411', 'request_id': 'req-154b301a-f695-47c5-88f6-f0c8db1bd6f8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:34.691169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f43c5cf-b6db-4ed6-bb4a-5ccf1604cec7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:34.693415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f43c5cf-b6db-4ed6-bb4a-5ccf1604cec7 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:34.701927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] The token cache is being invalidate because project 6a2311b2aa2340c5b5d2eab17913b411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:34.702371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:34.703545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-154b301a-f695-47c5-88f6-f0c8db1bd6f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:34.707286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 906/1818] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:34 2022] DELETE /identity/v3/projects/6a2311b2aa2340c5b5d2eab17913b411 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:34.713232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09faa7a5-091d-4a42-896d-78dcebc69227 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:34.713704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09faa7a5-091d-4a42-896d-78dcebc69227 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:34.714256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09faa7a5-091d-4a42-896d-78dcebc69227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:34.715115 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 913/1819] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:34.741982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-09faa7a5-091d-4a42-896d-78dcebc69227 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:34.743657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-09faa7a5-091d-4a42-896d-78dcebc69227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:34.804043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43c5cf-b6db-4ed6-bb4a-5ccf1604cec7 service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:34.811735 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:34.819792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43c5cf-b6db-4ed6-bb4a-5ccf1604cec7 service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:34.908640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 907/1820] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:34.912606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a083b-9f3f-4b9b-8381-8e42cea1637e None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:34.913335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a083b-9f3f-4b9b-8381-8e42cea1637e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:34.913828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a083b-9f3f-4b9b-8381-8e42cea1637e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:34.938884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-803a083b-9f3f-4b9b-8381-8e42cea1637e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:34.940509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-803a083b-9f3f-4b9b-8381-8e42cea1637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:34.949892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 908/1821] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:34.954052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a083b-9f3f-4b9b-8381-8e42cea1637e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:34.963543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:34.967285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:34.967285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:34.967285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] PATH_INFO: `/v3/users/e8c14b75059d4e3c8815d034067e84a0` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:34.967285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c14b75059d4e3c8815d034067e84a0)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:34.968929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:34.975528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a083b-9f3f-4b9b-8381-8e42cea1637e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:34.981796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba462b-c1bf-41b8-8dc0-845e40f3aebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:34.982947 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba462b-c1bf-41b8-8dc0-845e40f3aebd admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:34.983355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba462b-c1bf-41b8-8dc0-845e40f3aebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:34.983794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba462b-c1bf-41b8-8dc0-845e40f3aebd admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:34.995714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba462b-c1bf-41b8-8dc0-845e40f3aebd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:34.996035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba462b-c1bf-41b8-8dc0-845e40f3aebd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:34.997840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba462b-c1bf-41b8-8dc0-845e40f3aebd admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:35.031217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 909/1822] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:54:35.036081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba462b-c1bf-41b8-8dc0-845e40f3aebd admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:35.043627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c14b75059d4e3c8815d034067e84a0'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:35.052624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c14b75059d4e3c8815d034067e84a0'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:35.053413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c14b75059d4e3c8815d034067e84a0'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:35.054582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba462b-c1bf-41b8-8dc0-845e40f3aebd admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:35.063846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdd0e8-5b1d-4418-937c-88212d73e844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:35.064566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdd0e8-5b1d-4418-937c-88212d73e844 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.064854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdd0e8-5b1d-4418-937c-88212d73e844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.065186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdd0e8-5b1d-4418-937c-88212d73e844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.066237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdd0e8-5b1d-4418-937c-88212d73e844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:35.069120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c14b75059d4e3c8815d034067e84a0'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:35.074532 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdd0e8-5b1d-4418-937c-88212d73e844 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:35.093341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fd1f3da-58b0-4d4e-9b74-0ab351093253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c14b75059d4e3c8815d034067e84a0'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:35.105067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 914/1823] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:34 2022] DELETE /identity/v3/users/e8c14b75059d4e3c8815d034067e84a0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:54:35.114270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc51f5-a844-4cec-810a-291f3988e421 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.114569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc51f5-a844-4cec-810a-291f3988e421 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.115152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc51f5-a844-4cec-810a-291f3988e421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.126645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 910/1824] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:35 2022] POST /identity/v3/projects => generated 377 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:35.131986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdd0e8-5b1d-4418-937c-88212d73e844 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:35.134393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-95bdd0e8-5b1d-4418-937c-88212d73e844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:35.157614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a8cc51f5-a844-4cec-810a-291f3988e421 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:35.158905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a8cc51f5-a844-4cec-810a-291f3988e421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:35.226376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:35.230273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdd0e8-5b1d-4418-937c-88212d73e844 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:35.238559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29db00f-e2b9-4535-ae07-7671a3154ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:35.240359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29db00f-e2b9-4535-ae07-7671a3154ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.240648 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29db00f-e2b9-4535-ae07-7671a3154ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.241621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29db00f-e2b9-4535-ae07-7671a3154ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.242651 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29db00f-e2b9-4535-ae07-7671a3154ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:35.245321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29db00f-e2b9-4535-ae07-7671a3154ca9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:35.251476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 915/1825] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:35.259153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc51f5-a844-4cec-810a-291f3988e421 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:35.261279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a8cc51f5-a844-4cec-810a-291f3988e421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:35.271389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 911/1826] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:35 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:35.275632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29db00f-e2b9-4535-ae07-7671a3154ca9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:35.286579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29db00f-e2b9-4535-ae07-7671a3154ca9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:35.291270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83011d86-2015-4edd-a947-32876e963a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:35.291963 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011d86-2015-4edd-a947-32876e963a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.292234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011d86-2015-4edd-a947-32876e963a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.292510 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011d86-2015-4edd-a947-32876e963a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.293245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83011d86-2015-4edd-a947-32876e963a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:35.293924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:35.294862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83011d86-2015-4edd-a947-32876e963a9a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:35.306722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 912/1827] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:35.310354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83011d86-2015-4edd-a947-32876e963a9a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:35.320405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83011d86-2015-4edd-a947-32876e963a9a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:35.325442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befadf2-4b02-4627-b817-4a74dbb8b4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:35.326099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befadf2-4b02-4627-b817-4a74dbb8b4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.326415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befadf2-4b02-4627-b817-4a74dbb8b4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.326696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befadf2-4b02-4627-b817-4a74dbb8b4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.327446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befadf2-4b02-4627-b817-4a74dbb8b4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:35.328025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc51f5-a844-4cec-810a-291f3988e421 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:35.329043 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befadf2-4b02-4627-b817-4a74dbb8b4b8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:35.331670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f26222-3a8b-4b1d-bcd5-cfe3f88e1c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:35.332315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f26222-3a8b-4b1d-bcd5-cfe3f88e1c8b service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.332592 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f26222-3a8b-4b1d-bcd5-cfe3f88e1c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.332860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f26222-3a8b-4b1d-bcd5-cfe3f88e1c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.339905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 913/1828] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:35.342593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f26222-3a8b-4b1d-bcd5-cfe3f88e1c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:35.344063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f26222-3a8b-4b1d-bcd5-cfe3f88e1c8b service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:35.344191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befadf2-4b02-4627-b817-4a74dbb8b4b8 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:35.353702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 916/1829] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:35.353848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befadf2-4b02-4627-b817-4a74dbb8b4b8 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:35.359935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718b1ba-2a9b-4080-b3ef-9cec92f18d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:35.361115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718b1ba-2a9b-4080-b3ef-9cec92f18d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.361590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718b1ba-2a9b-4080-b3ef-9cec92f18d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.361868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718b1ba-2a9b-4080-b3ef-9cec92f18d9d admin admin] PATH_INFO: `/v3/projects/059c946aae6440d6b292327ae0a4a874/users/0a83557d4baf4280897c2e95a8937334/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.380198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718b1ba-2a9b-4080-b3ef-9cec92f18d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=059c946aae6440d6b292327ae0a4a874, user_id=0a83557d4baf4280897c2e95a8937334, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:35.382935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718b1ba-2a9b-4080-b3ef-9cec92f18d9d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:35.400645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 914/1830] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:35 2022] PUT /identity/v3/projects/059c946aae6440d6b292327ae0a4a874/users/0a83557d4baf4280897c2e95a8937334/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:54:35.413875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b9ed4-4dc1-4769-9681-b752060271be None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.414207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b9ed4-4dc1-4769-9681-b752060271be None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.414507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b9ed4-4dc1-4769-9681-b752060271be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.455496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-7c2b9ed4-4dc1-4769-9681-b752060271be None None] MFA Rules not processed for user `0a83557d4baf4280897c2e95a8937334`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:35.459161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7c2b9ed4-4dc1-4769-9681-b752060271be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:35.553466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 917/1831] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:35.558018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718b1ba-2a9b-4080-b3ef-9cec92f18d9d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:35.664766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718b1ba-2a9b-4080-b3ef-9cec92f18d9d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:35.687967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fb9f3-c4f3-43f5-855e-f7a052e1cd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:35.688678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fb9f3-c4f3-43f5-855e-f7a052e1cd86 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.688962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fb9f3-c4f3-43f5-855e-f7a052e1cd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.689237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fb9f3-c4f3-43f5-855e-f7a052e1cd86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.689952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fb9f3-c4f3-43f5-855e-f7a052e1cd86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:35.692232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fb9f3-c4f3-43f5-855e-f7a052e1cd86 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:35.749323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 915/1832] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:35 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:35.753421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b9ed4-4dc1-4769-9681-b752060271be None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:35.764607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b9ed4-4dc1-4769-9681-b752060271be None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:35.832009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19519940-1778-47e1-beca-01574ab65f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:35.832829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19519940-1778-47e1-beca-01574ab65f8e admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.833214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19519940-1778-47e1-beca-01574ab65f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.833592 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19519940-1778-47e1-beca-01574ab65f8e admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.835389 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19519940-1778-47e1-beca-01574ab65f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:35.837768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19519940-1778-47e1-beca-01574ab65f8e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:35.875525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 918/1833] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:35 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:35.879277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fb9f3-c4f3-43f5-855e-f7a052e1cd86 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:35.892742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fb9f3-c4f3-43f5-855e-f7a052e1cd86 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:35.900400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78434a1f-937a-4f6e-b6fd-5294046d2880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:35.902002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78434a1f-937a-4f6e-b6fd-5294046d2880 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.902241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78434a1f-937a-4f6e-b6fd-5294046d2880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.902531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78434a1f-937a-4f6e-b6fd-5294046d2880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.903672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78434a1f-937a-4f6e-b6fd-5294046d2880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:35.906514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78434a1f-937a-4f6e-b6fd-5294046d2880 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:35.928770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19519940-1778-47e1-beca-01574ab65f8e admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:35.936595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 916/1834] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:35.941797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19519940-1778-47e1-beca-01574ab65f8e admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:35.961177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589c926-69dc-4d6d-9113-edcc10c47c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:35.962368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589c926-69dc-4d6d-9113-edcc10c47c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:35.962766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589c926-69dc-4d6d-9113-edcc10c47c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:35.963241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589c926-69dc-4d6d-9113-edcc10c47c66 admin admin] PATH_INFO: `/v3/projects/7643802db0b143b3ab1d3acc7ffdffc6/users/32a82424e7794dcd8ab6acb8bf605461/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:35.991801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589c926-69dc-4d6d-9113-edcc10c47c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7643802db0b143b3ab1d3acc7ffdffc6, user_id=32a82424e7794dcd8ab6acb8bf605461, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:35.995117 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589c926-69dc-4d6d-9113-edcc10c47c66 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:36.031742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 919/1835] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:35 2022] PUT /identity/v3/projects/7643802db0b143b3ab1d3acc7ffdffc6/users/32a82424e7794dcd8ab6acb8bf605461/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:36.031954 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78434a1f-937a-4f6e-b6fd-5294046d2880 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:36.043703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78434a1f-937a-4f6e-b6fd-5294046d2880 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:36.150887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff36fc4-c305-45c0-a2ac-e1f5e715e5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:36.152521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff36fc4-c305-45c0-a2ac-e1f5e715e5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:36.152976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff36fc4-c305-45c0-a2ac-e1f5e715e5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:36.153347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff36fc4-c305-45c0-a2ac-e1f5e715e5c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:36.156965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff36fc4-c305-45c0-a2ac-e1f5e715e5c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:36.162689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff36fc4-c305-45c0-a2ac-e1f5e715e5c6 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:36.182595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589c926-69dc-4d6d-9113-edcc10c47c66 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:36.183727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 917/1836] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:36.209157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589c926-69dc-4d6d-9113-edcc10c47c66 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:36.215585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4875f2-0bc6-409f-83b5-04da273b13f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:36.218575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4875f2-0bc6-409f-83b5-04da273b13f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:36.220537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4875f2-0bc6-409f-83b5-04da273b13f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:36.220851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4875f2-0bc6-409f-83b5-04da273b13f9 admin admin] PATH_INFO: `/v3/projects/7643802db0b143b3ab1d3acc7ffdffc6/users/32a82424e7794dcd8ab6acb8bf605461/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:36.226124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4875f2-0bc6-409f-83b5-04da273b13f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7643802db0b143b3ab1d3acc7ffdffc6, user_id=32a82424e7794dcd8ab6acb8bf605461, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:36.230024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4875f2-0bc6-409f-83b5-04da273b13f9 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:36.253590 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 920/1837] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:36 2022] PUT /identity/v3/projects/7643802db0b143b3ab1d3acc7ffdffc6/users/32a82424e7794dcd8ab6acb8bf605461/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:36.256910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54135562-bf75-4a92-a624-b34f097339de None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:36.257464 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54135562-bf75-4a92-a624-b34f097339de None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:36.257934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54135562-bf75-4a92-a624-b34f097339de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:36.259296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4875f2-0bc6-409f-83b5-04da273b13f9 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:36.266140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9e4875f2-0bc6-409f-83b5-04da273b13f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:36.287937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:36.289692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:36.306527 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-54135562-bf75-4a92-a624-b34f097339de None None] MFA Rules not processed for user `32a82424e7794dcd8ab6acb8bf605461`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:36.308049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-54135562-bf75-4a92-a624-b34f097339de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:36.442360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4875f2-0bc6-409f-83b5-04da273b13f9 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:36.451801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:36.453165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:36.453581 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:36.454037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] PATH_INFO: `/v3/projects/7b4c5894a19c46c9a4150fc26b0f8ced` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:36.455620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4c5894a19c46c9a4150fc26b0f8ced)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:36.459355 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:36.482589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 918/1838] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:36.497060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54135562-bf75-4a92-a624-b34f097339de None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:36.500532 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-54135562-bf75-4a92-a624-b34f097339de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:36.549008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4c5894a19c46c9a4150fc26b0f8ced', 'request_id': 'req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:36.557554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4c5894a19c46c9a4150fc26b0f8ced', 'request_id': 'req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:36.567362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4c5894a19c46c9a4150fc26b0f8ced', 'request_id': 'req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:36.585781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54135562-bf75-4a92-a624-b34f097339de None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:36.589036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] The token cache is being invalidate because project 7b4c5894a19c46c9a4150fc26b0f8ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:36.589456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:36.590246 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:36.592220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 921/1839] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:36 2022] DELETE /identity/v3/projects/7b4c5894a19c46c9a4150fc26b0f8ced => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:36.596175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:36.597651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:36.676348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2afa80-2fd9-4f91-a4f6-486c465f0220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:36.677339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2afa80-2fd9-4f91-a4f6-486c465f0220 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:36.677629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2afa80-2fd9-4f91-a4f6-486c465f0220 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:36.677906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2afa80-2fd9-4f91-a4f6-486c465f0220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:36.681747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ab2afa80-2fd9-4f91-a4f6-486c465f0220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:36.707913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c326cd8-b4d9-4731-b735-fe8c06b23bf3 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:36.723062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:36.724015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:36.727538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:36.727880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] PATH_INFO: `/v3/users/0761eeeda9894151bb595cf92a491732` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:36.730712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0761eeeda9894151bb595cf92a491732)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:36.733820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:36.787462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2afa80-2fd9-4f91-a4f6-486c465f0220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:36.789043 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2afa80-2fd9-4f91-a4f6-486c465f0220 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:36.795156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:36.804327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0761eeeda9894151bb595cf92a491732'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:36.816167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 919/1840] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:36.817942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0761eeeda9894151bb595cf92a491732'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:36.819810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0761eeeda9894151bb595cf92a491732'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:36.852437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0761eeeda9894151bb595cf92a491732'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:36.881174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bf3a6f89-b73d-4b79-aa59-dc864fcd8ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0761eeeda9894151bb595cf92a491732'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:36.888310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 922/1841] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:36 2022] DELETE /identity/v3/users/0761eeeda9894151bb595cf92a491732 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:37.890496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2afa80-2fd9-4f91-a4f6-486c465f0220 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:37.903976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2afa80-2fd9-4f91-a4f6-486c465f0220 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:37.911294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:37.996039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:37.997589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:37.998014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:37.998427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] PATH_INFO: `/v3/projects/d833e893e28c4e60831a10a67098b92e` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:37.999917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d833e893e28c4e60831a10a67098b92e)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:38.002487 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:38.075185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd833e893e28c4e60831a10a67098b92e', 'request_id': 'req-28c89e0f-0f6c-4821-881a-8703d7e72590'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:38.083535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd833e893e28c4e60831a10a67098b92e', 'request_id': 'req-28c89e0f-0f6c-4821-881a-8703d7e72590'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:38.089839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd833e893e28c4e60831a10a67098b92e', 'request_id': 'req-28c89e0f-0f6c-4821-881a-8703d7e72590'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:38.106130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] The token cache is being invalidate because project d833e893e28c4e60831a10a67098b92e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:38.106631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:38.107874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-28c89e0f-0f6c-4821-881a-8703d7e72590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:38.110014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 920/1842] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:37 2022] DELETE /identity/v3/projects/d833e893e28c4e60831a10a67098b92e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:38.115018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d4b3e-0bbd-40cf-9ecf-77ae271759f6 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.115906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d4b3e-0bbd-40cf-9ecf-77ae271759f6 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.116903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d4b3e-0bbd-40cf-9ecf-77ae271759f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.136757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-921d4b3e-0bbd-40cf-9ecf-77ae271759f6 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:38.137980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-921d4b3e-0bbd-40cf-9ecf-77ae271759f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:38.204259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 923/1843] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:38.212140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb5711-46e3-4922-a5cf-83acb1ad75ae None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.212428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb5711-46e3-4922-a5cf-83acb1ad75ae None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.212703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb5711-46e3-4922-a5cf-83acb1ad75ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.231377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-f3cb5711-46e3-4922-a5cf-83acb1ad75ae None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:38.232718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f3cb5711-46e3-4922-a5cf-83acb1ad75ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:38.243011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 921/1844] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:38.245855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d4b3e-0bbd-40cf-9ecf-77ae271759f6 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:38.255981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d4b3e-0bbd-40cf-9ecf-77ae271759f6 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:38.260853 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c1b11-14ff-4d67-9b18-8e174f3e6ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:38.261550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c1b11-14ff-4d67-9b18-8e174f3e6ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.261834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c1b11-14ff-4d67-9b18-8e174f3e6ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.262114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c1b11-14ff-4d67-9b18-8e174f3e6ae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.262857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c1b11-14ff-4d67-9b18-8e174f3e6ae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:38.263166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c1b11-14ff-4d67-9b18-8e174f3e6ae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:38.264962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c1b11-14ff-4d67-9b18-8e174f3e6ae8 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:38.277775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 924/1845] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:38.282501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb5711-46e3-4922-a5cf-83acb1ad75ae None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:38.292061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb5711-46e3-4922-a5cf-83acb1ad75ae None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:38.297098 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4e8f7-9178-44aa-a953-1bcbf06b8586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:38.297929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4e8f7-9178-44aa-a953-1bcbf06b8586 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.298379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4e8f7-9178-44aa-a953-1bcbf06b8586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.298656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4e8f7-9178-44aa-a953-1bcbf06b8586 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.299589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4e8f7-9178-44aa-a953-1bcbf06b8586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:38.302238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4e8f7-9178-44aa-a953-1bcbf06b8586 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:38.332314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 922/1846] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:38 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:54:38.336646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c1b11-14ff-4d67-9b18-8e174f3e6ae8 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:38.347572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c1b11-14ff-4d67-9b18-8e174f3e6ae8 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:38.408432 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90fe36-0d65-439d-8350-45e65db82c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:38.409267 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90fe36-0d65-439d-8350-45e65db82c88 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.409557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90fe36-0d65-439d-8350-45e65db82c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.409834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90fe36-0d65-439d-8350-45e65db82c88 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.410524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90fe36-0d65-439d-8350-45e65db82c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:38.412535 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90fe36-0d65-439d-8350-45e65db82c88 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:38.438019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 925/1847] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:38 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:38.442285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4e8f7-9178-44aa-a953-1bcbf06b8586 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:38.453474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4e8f7-9178-44aa-a953-1bcbf06b8586 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:38.458902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b922733-136e-4025-a3e0-d5303d446fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:38.459627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b922733-136e-4025-a3e0-d5303d446fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.460036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b922733-136e-4025-a3e0-d5303d446fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.460315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b922733-136e-4025-a3e0-d5303d446fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.461060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b922733-136e-4025-a3e0-d5303d446fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:38.462764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b922733-136e-4025-a3e0-d5303d446fa2 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:38.476033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 923/1848] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:38.480311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90fe36-0d65-439d-8350-45e65db82c88 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:38.491545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90fe36-0d65-439d-8350-45e65db82c88 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:38.497835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398cfd1-5a23-40d3-b2c2-bb008a34abfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:38.498676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398cfd1-5a23-40d3-b2c2-bb008a34abfc admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.498949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398cfd1-5a23-40d3-b2c2-bb008a34abfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.499443 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398cfd1-5a23-40d3-b2c2-bb008a34abfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.500136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398cfd1-5a23-40d3-b2c2-bb008a34abfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:38.507046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398cfd1-5a23-40d3-b2c2-bb008a34abfc admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:38.517583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 926/1849] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:38.518061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b922733-136e-4025-a3e0-d5303d446fa2 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:38.529066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b922733-136e-4025-a3e0-d5303d446fa2 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:38.534843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bd79-c56c-401a-b40e-6cc25eaf9090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:38.535778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bd79-c56c-401a-b40e-6cc25eaf9090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.536057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bd79-c56c-401a-b40e-6cc25eaf9090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.536339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bd79-c56c-401a-b40e-6cc25eaf9090 admin admin] PATH_INFO: `/v3/projects/2ae600473f4f4e4eb4e1615b952580d4/users/68e9009274054965819322279a4b35f7/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.551662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9bd79-c56c-401a-b40e-6cc25eaf9090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae600473f4f4e4eb4e1615b952580d4, user_id=68e9009274054965819322279a4b35f7, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:38.555288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9bd79-c56c-401a-b40e-6cc25eaf9090 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:38.573772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 924/1850] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:38 2022] PUT /identity/v3/projects/2ae600473f4f4e4eb4e1615b952580d4/users/68e9009274054965819322279a4b35f7/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:54:38.585331 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5837dec-b527-44eb-af96-6a8cb3fa0228 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.585680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5837dec-b527-44eb-af96-6a8cb3fa0228 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.585984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5837dec-b527-44eb-af96-6a8cb3fa0228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.614318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-e5837dec-b527-44eb-af96-6a8cb3fa0228 None None] MFA Rules not processed for user `68e9009274054965819322279a4b35f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:38.615589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e5837dec-b527-44eb-af96-6a8cb3fa0228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:38.627654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:38.682973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 927/1851] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:38.688054 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bd79-c56c-401a-b40e-6cc25eaf9090 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:38.732640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:38.768539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bd79-c56c-401a-b40e-6cc25eaf9090 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:38.783997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16253807-7116-41e0-a556-fa409c3a5777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:38.789210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16253807-7116-41e0-a556-fa409c3a5777 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.789210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16253807-7116-41e0-a556-fa409c3a5777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.789210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16253807-7116-41e0-a556-fa409c3a5777 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.789210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16253807-7116-41e0-a556-fa409c3a5777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:38.789210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16253807-7116-41e0-a556-fa409c3a5777 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:38.880224 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 925/1852] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:38 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:54:38.881294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5837dec-b527-44eb-af96-6a8cb3fa0228 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:38.899588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5837dec-b527-44eb-af96-6a8cb3fa0228 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:38.972973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec48a-9767-43c0-a0c9-743d6233abb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:38.973589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec48a-9767-43c0-a0c9-743d6233abb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:38.973873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec48a-9767-43c0-a0c9-743d6233abb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:38.974144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec48a-9767-43c0-a0c9-743d6233abb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:38.974856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ec48a-9767-43c0-a0c9-743d6233abb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:38.977157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ec48a-9767-43c0-a0c9-743d6233abb3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:39.002830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 928/1853] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:38 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:54:39.007175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16253807-7116-41e0-a556-fa409c3a5777 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:39.020990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16253807-7116-41e0-a556-fa409c3a5777 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:39.026054 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc032-4de3-40b0-a2f1-8a4b2206f204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:39.026745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc032-4de3-40b0-a2f1-8a4b2206f204 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:39.027036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc032-4de3-40b0-a2f1-8a4b2206f204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:39.027337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc032-4de3-40b0-a2f1-8a4b2206f204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:39.028108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edc032-4de3-40b0-a2f1-8a4b2206f204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:39.029792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edc032-4de3-40b0-a2f1-8a4b2206f204 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:39.041033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 926/1854] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:39.045191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec48a-9767-43c0-a0c9-743d6233abb3 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:39.056172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec48a-9767-43c0-a0c9-743d6233abb3 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:39.061117 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade899b0-c3aa-44f7-9475-52d3b208e9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:39.062102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade899b0-c3aa-44f7-9475-52d3b208e9f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:39.062379 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade899b0-c3aa-44f7-9475-52d3b208e9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:39.062649 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade899b0-c3aa-44f7-9475-52d3b208e9f8 admin admin] PATH_INFO: `/v3/projects/03e3951e19fb482cb7bd7e7cf04d5180/users/9cb5991968664de8a94494d9619df774/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:39.077613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade899b0-c3aa-44f7-9475-52d3b208e9f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e3951e19fb482cb7bd7e7cf04d5180, user_id=9cb5991968664de8a94494d9619df774, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:39.080374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade899b0-c3aa-44f7-9475-52d3b208e9f8 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:39.095610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 929/1855] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:39 2022] PUT /identity/v3/projects/03e3951e19fb482cb7bd7e7cf04d5180/users/9cb5991968664de8a94494d9619df774/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:39.099649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc032-4de3-40b0-a2f1-8a4b2206f204 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:39.110602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc032-4de3-40b0-a2f1-8a4b2206f204 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:39.169150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cae9d3-66a0-4f9d-9e15-f68f5c7a897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:39.169899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cae9d3-66a0-4f9d-9e15-f68f5c7a897f admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:39.170282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cae9d3-66a0-4f9d-9e15-f68f5c7a897f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:39.170625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cae9d3-66a0-4f9d-9e15-f68f5c7a897f admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:39.171472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cae9d3-66a0-4f9d-9e15-f68f5c7a897f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:39.173172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cae9d3-66a0-4f9d-9e15-f68f5c7a897f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:39.187030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 927/1856] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:39.191225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade899b0-c3aa-44f7-9475-52d3b208e9f8 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:39.203325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade899b0-c3aa-44f7-9475-52d3b208e9f8 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:39.230109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747aef6-5441-4894-a4fd-d493ca862baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:39.231459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747aef6-5441-4894-a4fd-d493ca862baf admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:39.231744 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747aef6-5441-4894-a4fd-d493ca862baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:39.232045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747aef6-5441-4894-a4fd-d493ca862baf admin admin] PATH_INFO: `/v3/projects/03e3951e19fb482cb7bd7e7cf04d5180/users/9cb5991968664de8a94494d9619df774/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:39.235341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747aef6-5441-4894-a4fd-d493ca862baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e3951e19fb482cb7bd7e7cf04d5180, user_id=9cb5991968664de8a94494d9619df774, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:39.237597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747aef6-5441-4894-a4fd-d493ca862baf admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:39.257761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 930/1857] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:39 2022] PUT /identity/v3/projects/03e3951e19fb482cb7bd7e7cf04d5180/users/9cb5991968664de8a94494d9619df774/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:39.262748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52c8bd-29c4-4c71-afe3-3506313d5f97 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:39.263214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52c8bd-29c4-4c71-afe3-3506313d5f97 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:39.263649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52c8bd-29c4-4c71-afe3-3506313d5f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:39.295549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4e52c8bd-29c4-4c71-afe3-3506313d5f97 None None] MFA Rules not processed for user `9cb5991968664de8a94494d9619df774`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:39.296975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4e52c8bd-29c4-4c71-afe3-3506313d5f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:39.372570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 928/1858] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:39.383979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747aef6-5441-4894-a4fd-d493ca862baf admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:39.385748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5747aef6-5441-4894-a4fd-d493ca862baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:39.448786 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747aef6-5441-4894-a4fd-d493ca862baf admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:39.453895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f2838-3e9a-4b99-aecb-67cffcd4e229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:39.454520 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f2838-3e9a-4b99-aecb-67cffcd4e229 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:39.454799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f2838-3e9a-4b99-aecb-67cffcd4e229 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:39.455072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f2838-3e9a-4b99-aecb-67cffcd4e229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:39.464706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f2838-3e9a-4b99-aecb-67cffcd4e229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:39.466156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f2838-3e9a-4b99-aecb-67cffcd4e229 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:39.475805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 931/1859] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:39 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:39.558011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52c8bd-29c4-4c71-afe3-3506313d5f97 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:39.567992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52c8bd-29c4-4c71-afe3-3506313d5f97 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:39.571851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46299e62-bfd5-414c-8283-1bf53a93c605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:39.572500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46299e62-bfd5-414c-8283-1bf53a93c605 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:39.572780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46299e62-bfd5-414c-8283-1bf53a93c605 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:39.573053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46299e62-bfd5-414c-8283-1bf53a93c605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:39.600847 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:39.635821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46299e62-bfd5-414c-8283-1bf53a93c605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:39.637284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46299e62-bfd5-414c-8283-1bf53a93c605 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:39.646719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 929/1860] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:42.442453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f2838-3e9a-4b99-aecb-67cffcd4e229 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:42.455644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:42.550588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f2838-3e9a-4b99-aecb-67cffcd4e229 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:42.561539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:42.563062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:42.563493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:42.564103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] PATH_INFO: `/v3/users/68e9009274054965819322279a4b35f7` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:42.565412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e9009274054965819322279a4b35f7)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:42.567390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:42.642890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e9009274054965819322279a4b35f7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:42.654674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e9009274054965819322279a4b35f7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:42.655475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e9009274054965819322279a4b35f7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:42.664932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e9009274054965819322279a4b35f7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:42.682608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e9009274054965819322279a4b35f7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:42.694372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 932/1861] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:42 2022] DELETE /identity/v3/users/68e9009274054965819322279a4b35f7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:42.698624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d4d1e-c149-4daf-a88c-01c5644b5d3c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:42.699607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d4d1e-c149-4daf-a88c-01c5644b5d3c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:42.700095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d4d1e-c149-4daf-a88c-01c5644b5d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:42.741296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-fe3d4d1e-c149-4daf-a88c-01c5644b5d3c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:42.742992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fe3d4d1e-c149-4daf-a88c-01c5644b5d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:42.823299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 930/1862] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:42.843729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:42.845197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:42.936599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9437f7-2d96-40c9-be4e-fbcac28015e8 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:42.940571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3937b9-4129-482f-b5c4-d3ca1aae165e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:42.941105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3937b9-4129-482f-b5c4-d3ca1aae165e service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:42.941388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3937b9-4129-482f-b5c4-d3ca1aae165e service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:42.941663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3937b9-4129-482f-b5c4-d3ca1aae165e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:42.953900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3937b9-4129-482f-b5c4-d3ca1aae165e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:42.955998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3937b9-4129-482f-b5c4-d3ca1aae165e service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:42.967901 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 933/1863] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:43.258011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d4d1e-c149-4daf-a88c-01c5644b5d3c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:43.259578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fe3d4d1e-c149-4daf-a88c-01c5644b5d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:43.273241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d4d1e-c149-4daf-a88c-01c5644b5d3c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:43.278515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:43.279681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:43.280009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:43.280273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] PATH_INFO: `/v3/projects/2ae600473f4f4e4eb4e1615b952580d4` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:43.281564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae600473f4f4e4eb4e1615b952580d4)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:43.283588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:43.337014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae600473f4f4e4eb4e1615b952580d4', 'request_id': 'req-506134d9-95db-44f1-8e53-92c8ef1e5735'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:43.343290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae600473f4f4e4eb4e1615b952580d4', 'request_id': 'req-506134d9-95db-44f1-8e53-92c8ef1e5735'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:43.348919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae600473f4f4e4eb4e1615b952580d4', 'request_id': 'req-506134d9-95db-44f1-8e53-92c8ef1e5735'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:43.363854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] The token cache is being invalidate because project 2ae600473f4f4e4eb4e1615b952580d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:43.364143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:43.364762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:43.366411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 931/1864] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:43 2022] DELETE /identity/v3/projects/2ae600473f4f4e4eb4e1615b952580d4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:43.370579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3937b9-4129-482f-b5c4-d3ca1aae165e service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:43.380448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3937b9-4129-482f-b5c4-d3ca1aae165e service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:43.413954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:43.445091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:43.445980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:43.446261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:43.446532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] PATH_INFO: `/v3/users/9cb5991968664de8a94494d9619df774` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:43.447773 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb5991968664de8a94494d9619df774)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:43.449719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:43.495489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb5991968664de8a94494d9619df774'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:43.502436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb5991968664de8a94494d9619df774'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:43.503120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb5991968664de8a94494d9619df774'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:43.513998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb5991968664de8a94494d9619df774'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:43.527014 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d7f411a9-e04e-44d9-9b65-7d9c29a98e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb5991968664de8a94494d9619df774'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:43.533236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 934/1865] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:43 2022] DELETE /identity/v3/users/9cb5991968664de8a94494d9619df774 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:43.996481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:44.000956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:44.021029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506134d9-95db-44f1-8e53-92c8ef1e5735 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:44.082515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:44.095229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:44.097423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.097996 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.098327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] PATH_INFO: `/v3/projects/03e3951e19fb482cb7bd7e7cf04d5180` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.099985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e3951e19fb482cb7bd7e7cf04d5180)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:44.102615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:44.174441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e3951e19fb482cb7bd7e7cf04d5180', 'request_id': 'req-bdc8de0a-877c-484a-85ef-ddb923ff1532'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:44.186546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e3951e19fb482cb7bd7e7cf04d5180', 'request_id': 'req-bdc8de0a-877c-484a-85ef-ddb923ff1532'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:44.194031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e3951e19fb482cb7bd7e7cf04d5180', 'request_id': 'req-bdc8de0a-877c-484a-85ef-ddb923ff1532'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:44.210238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] The token cache is being invalidate because project 03e3951e19fb482cb7bd7e7cf04d5180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:44.211317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:44.211317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdc8de0a-877c-484a-85ef-ddb923ff1532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:44.213362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 932/1866] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:43 2022] DELETE /identity/v3/projects/03e3951e19fb482cb7bd7e7cf04d5180 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:44.223518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334c6dc-7673-44a0-8264-70580bcdd5d0 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.223998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334c6dc-7673-44a0-8264-70580bcdd5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.225257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334c6dc-7673-44a0-8264-70580bcdd5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.244063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b334c6dc-7673-44a0-8264-70580bcdd5d0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:44.245287 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b334c6dc-7673-44a0-8264-70580bcdd5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:44.308063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 935/1867] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:44.315973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb731d1b-c9c3-4d55-bc36-f734396c4a69 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.316388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb731d1b-c9c3-4d55-bc36-f734396c4a69 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.316689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb731d1b-c9c3-4d55-bc36-f734396c4a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.337766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-bb731d1b-c9c3-4d55-bc36-f734396c4a69 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:44.338991 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-bb731d1b-c9c3-4d55-bc36-f734396c4a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:44.347169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 933/1868] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:44.351734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334c6dc-7673-44a0-8264-70580bcdd5d0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:44.362579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334c6dc-7673-44a0-8264-70580bcdd5d0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:44.367474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264955c6-23c0-4d75-a47c-c2a8011d8745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:44.368150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264955c6-23c0-4d75-a47c-c2a8011d8745 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.368431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264955c6-23c0-4d75-a47c-c2a8011d8745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.368703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264955c6-23c0-4d75-a47c-c2a8011d8745 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.369435 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264955c6-23c0-4d75-a47c-c2a8011d8745 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:44.369739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264955c6-23c0-4d75-a47c-c2a8011d8745 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:44.371445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264955c6-23c0-4d75-a47c-c2a8011d8745 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:44.389169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 936/1869] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:44.390453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb731d1b-c9c3-4d55-bc36-f734396c4a69 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:44.400142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb731d1b-c9c3-4d55-bc36-f734396c4a69 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:44.405093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbce518-26e8-4cee-aef3-ad9e5500790a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:44.405771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbce518-26e8-4cee-aef3-ad9e5500790a admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.406183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbce518-26e8-4cee-aef3-ad9e5500790a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.406488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbce518-26e8-4cee-aef3-ad9e5500790a admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.407182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbce518-26e8-4cee-aef3-ad9e5500790a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:44.409135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbce518-26e8-4cee-aef3-ad9e5500790a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:44.445328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 934/1870] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:44 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:44.446644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264955c6-23c0-4d75-a47c-c2a8011d8745 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:44.457035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264955c6-23c0-4d75-a47c-c2a8011d8745 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:44.468277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a6f2-bfd9-41f9-b650-d6705a395c52 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.468688 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a6f2-bfd9-41f9-b650-d6705a395c52 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.469016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a6f2-bfd9-41f9-b650-d6705a395c52 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.470587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 935/1871] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 05:54:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:54:44.484216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00c888-4163-462d-9cd5-2b1c4931936c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.484636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00c888-4163-462d-9cd5-2b1c4931936c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.485059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00c888-4163-462d-9cd5-2b1c4931936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.505195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ea00c888-4163-462d-9cd5-2b1c4931936c None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:44.506856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ea00c888-4163-462d-9cd5-2b1c4931936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:44.521424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:44.553050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8c701-a72a-47f1-9ddd-60923ec4ef03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:44.553857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8c701-a72a-47f1-9ddd-60923ec4ef03 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.554239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8c701-a72a-47f1-9ddd-60923ec4ef03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.555668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8c701-a72a-47f1-9ddd-60923ec4ef03 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.556499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8c701-a72a-47f1-9ddd-60923ec4ef03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:44.558669 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8c701-a72a-47f1-9ddd-60923ec4ef03 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:44.598866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 937/1872] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:44 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:44.602945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8c701-a72a-47f1-9ddd-60923ec4ef03 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:44.609618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 936/1873] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 05:54:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:44.614781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8c701-a72a-47f1-9ddd-60923ec4ef03 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:44.620853 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a09da-eaaf-4d19-8b28-d66de7afd9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:44.621617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a09da-eaaf-4d19-8b28-d66de7afd9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.621841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a09da-eaaf-4d19-8b28-d66de7afd9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.622166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a09da-eaaf-4d19-8b28-d66de7afd9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.623025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a09da-eaaf-4d19-8b28-d66de7afd9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:44.624850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a09da-eaaf-4d19-8b28-d66de7afd9b6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:44.637917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 938/1874] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:44.642192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00c888-4163-462d-9cd5-2b1c4931936c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:44.657393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a09da-eaaf-4d19-8b28-d66de7afd9b6 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:44.659326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-fb5a09da-eaaf-4d19-8b28-d66de7afd9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:44.669531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00c888-4163-462d-9cd5-2b1c4931936c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:44.676396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de73fe-4fc9-42a9-9212-949e7b57f76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:44.677648 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de73fe-4fc9-42a9-9212-949e7b57f76a admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.677648 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de73fe-4fc9-42a9-9212-949e7b57f76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.678061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de73fe-4fc9-42a9-9212-949e7b57f76a admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.679045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de73fe-4fc9-42a9-9212-949e7b57f76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:44.681218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de73fe-4fc9-42a9-9212-949e7b57f76a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:44.684595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a09da-eaaf-4d19-8b28-d66de7afd9b6 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:44.693431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f7e8f-41ae-48cf-8e62-0c08aeccf8c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:44.694633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f7e8f-41ae-48cf-8e62-0c08aeccf8c8 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.694923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f7e8f-41ae-48cf-8e62-0c08aeccf8c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.695827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f7e8f-41ae-48cf-8e62-0c08aeccf8c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.696599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 937/1875] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:44.700788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de73fe-4fc9-42a9-9212-949e7b57f76a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:44.716126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f7e8f-41ae-48cf-8e62-0c08aeccf8c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:44.718196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f7e8f-41ae-48cf-8e62-0c08aeccf8c8 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:44.722266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de73fe-4fc9-42a9-9212-949e7b57f76a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:44.728809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91c3fd-3987-4377-8adf-5ece39db18fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:44.730525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 939/1876] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:44.731120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91c3fd-3987-4377-8adf-5ece39db18fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.731637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91c3fd-3987-4377-8adf-5ece39db18fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.732305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91c3fd-3987-4377-8adf-5ece39db18fa admin admin] PATH_INFO: `/v3/projects/45b26c38ff5444dd91998771eebfbb10/users/7a8480ae60ca4e68a5258eb4cabe6348/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.757345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91c3fd-3987-4377-8adf-5ece39db18fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=45b26c38ff5444dd91998771eebfbb10, user_id=7a8480ae60ca4e68a5258eb4cabe6348, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:44.762221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91c3fd-3987-4377-8adf-5ece39db18fa admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:44.796758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 938/1877] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:44 2022] PUT /identity/v3/projects/45b26c38ff5444dd91998771eebfbb10/users/7a8480ae60ca4e68a5258eb4cabe6348/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:44.807261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b36c33-60ca-4d28-8f72-a4fb0dd32869 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:44.807559 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b36c33-60ca-4d28-8f72-a4fb0dd32869 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:44.807935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b36c33-60ca-4d28-8f72-a4fb0dd32869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:44.854397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91c3fd-3987-4377-8adf-5ece39db18fa admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:44.857919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1b91c3fd-3987-4377-8adf-5ece39db18fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:44.880359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:44.898440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-57b36c33-60ca-4d28-8f72-a4fb0dd32869 None None] MFA Rules not processed for user `7a8480ae60ca4e68a5258eb4cabe6348`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:44.900844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-57b36c33-60ca-4d28-8f72-a4fb0dd32869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:45.025951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91c3fd-3987-4377-8adf-5ece39db18fa admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:45.041064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4bb5d-3079-48dc-b79d-d9f32063eca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:45.041845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4bb5d-3079-48dc-b79d-d9f32063eca0 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:45.042192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4bb5d-3079-48dc-b79d-d9f32063eca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:45.042535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4bb5d-3079-48dc-b79d-d9f32063eca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:45.051284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ffd4bb5d-3079-48dc-b79d-d9f32063eca0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:45.102980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 940/1878] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:45.108108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b36c33-60ca-4d28-8f72-a4fb0dd32869 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:45.190319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4bb5d-3079-48dc-b79d-d9f32063eca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:45.191924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4bb5d-3079-48dc-b79d-d9f32063eca0 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:45.211323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 939/1879] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:45.228203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b36c33-60ca-4d28-8f72-a4fb0dd32869 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:45.234275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01da297-7682-4e90-a970-cb0f9a4b5790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:45.235239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01da297-7682-4e90-a970-cb0f9a4b5790 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:45.236342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01da297-7682-4e90-a970-cb0f9a4b5790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:45.237342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01da297-7682-4e90-a970-cb0f9a4b5790 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:45.238418 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01da297-7682-4e90-a970-cb0f9a4b5790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:45.240577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01da297-7682-4e90-a970-cb0f9a4b5790 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:45.302695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 941/1880] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:45 2022] POST /identity/v3/projects => generated 367 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:45.306883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4bb5d-3079-48dc-b79d-d9f32063eca0 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:45.322168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4bb5d-3079-48dc-b79d-d9f32063eca0 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:45.491116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2bb8f-bf73-4f46-8546-3537c22b9f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:45.492294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2bb8f-bf73-4f46-8546-3537c22b9f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:45.494677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2bb8f-bf73-4f46-8546-3537c22b9f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:45.495926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2bb8f-bf73-4f46-8546-3537c22b9f88 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:45.496872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2bb8f-bf73-4f46-8546-3537c22b9f88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:45.503851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2bb8f-bf73-4f46-8546-3537c22b9f88 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:45.559182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 940/1881] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:45 2022] POST /identity/v3/users => generated 290 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:45.563600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01da297-7682-4e90-a970-cb0f9a4b5790 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:45.583463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01da297-7682-4e90-a970-cb0f9a4b5790 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:45.602414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce573b3-de3f-44e1-9e5a-e0142cdbc78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:45.603671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce573b3-de3f-44e1-9e5a-e0142cdbc78d admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:45.605067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce573b3-de3f-44e1-9e5a-e0142cdbc78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:45.605769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce573b3-de3f-44e1-9e5a-e0142cdbc78d admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:45.609996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce573b3-de3f-44e1-9e5a-e0142cdbc78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:45.616934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce573b3-de3f-44e1-9e5a-e0142cdbc78d admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:45.650856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 942/1882] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:45.652259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2bb8f-bf73-4f46-8546-3537c22b9f88 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:45.672273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2bb8f-bf73-4f46-8546-3537c22b9f88 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:45.688936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732acfd-d6ac-4a50-8250-98ad1d3abec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:45.689873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732acfd-d6ac-4a50-8250-98ad1d3abec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:45.690382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732acfd-d6ac-4a50-8250-98ad1d3abec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:45.690907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732acfd-d6ac-4a50-8250-98ad1d3abec5 admin admin] PATH_INFO: `/v3/projects/bbe4a412dbda4a0ea571f82d62f45d33/users/7bb590bb03364f06b30bae47343a31cd/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:45.726103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732acfd-d6ac-4a50-8250-98ad1d3abec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe4a412dbda4a0ea571f82d62f45d33, user_id=7bb590bb03364f06b30bae47343a31cd, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:45.738084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732acfd-d6ac-4a50-8250-98ad1d3abec5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:45.778927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 941/1883] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:45 2022] PUT /identity/v3/projects/bbe4a412dbda4a0ea571f82d62f45d33/users/7bb590bb03364f06b30bae47343a31cd/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:45.785568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce573b3-de3f-44e1-9e5a-e0142cdbc78d admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:45.795908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce573b3-de3f-44e1-9e5a-e0142cdbc78d admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:45.801745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:45.890873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd725-39a0-4ab7-9fc0-11d7a56be429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:45.891924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd725-39a0-4ab7-9fc0-11d7a56be429 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:45.892476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd725-39a0-4ab7-9fc0-11d7a56be429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:45.892943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd725-39a0-4ab7-9fc0-11d7a56be429 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:45.893821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfd725-39a0-4ab7-9fc0-11d7a56be429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:45.895705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfd725-39a0-4ab7-9fc0-11d7a56be429 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:45.919368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732acfd-d6ac-4a50-8250-98ad1d3abec5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:45.922412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 943/1884] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:45.934446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732acfd-d6ac-4a50-8250-98ad1d3abec5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:45.941119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06b649-50fd-49d0-ad5b-c79d3892dfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:45.944934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06b649-50fd-49d0-ad5b-c79d3892dfa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:45.944934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06b649-50fd-49d0-ad5b-c79d3892dfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:45.945311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06b649-50fd-49d0-ad5b-c79d3892dfa7 admin admin] PATH_INFO: `/v3/projects/bbe4a412dbda4a0ea571f82d62f45d33/users/7bb590bb03364f06b30bae47343a31cd/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:45.948181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06b649-50fd-49d0-ad5b-c79d3892dfa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe4a412dbda4a0ea571f82d62f45d33, user_id=7bb590bb03364f06b30bae47343a31cd, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:45.953962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06b649-50fd-49d0-ad5b-c79d3892dfa7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:45.985539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967f3b1-cdb4-4507-bfb2-8913c28de74b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:45.985938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967f3b1-cdb4-4507-bfb2-8913c28de74b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:45.986430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967f3b1-cdb4-4507-bfb2-8913c28de74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:45.988004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 942/1885] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:45 2022] PUT /identity/v3/projects/bbe4a412dbda4a0ea571f82d62f45d33/users/7bb590bb03364f06b30bae47343a31cd/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:46.032969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-0967f3b1-cdb4-4507-bfb2-8913c28de74b None None] MFA Rules not processed for user `7bb590bb03364f06b30bae47343a31cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:46.034637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0967f3b1-cdb4-4507-bfb2-8913c28de74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:46.164366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 944/1886] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:45 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:46.175130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06b649-50fd-49d0-ad5b-c79d3892dfa7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:46.193211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:46.254979 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06b649-50fd-49d0-ad5b-c79d3892dfa7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:46.258964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8588d37-a2d9-44b6-b7c1-80a2aa594315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:46.259676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8588d37-a2d9-44b6-b7c1-80a2aa594315 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:46.259959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8588d37-a2d9-44b6-b7c1-80a2aa594315 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:46.260235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8588d37-a2d9-44b6-b7c1-80a2aa594315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:46.270334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8588d37-a2d9-44b6-b7c1-80a2aa594315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:46.271824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8588d37-a2d9-44b6-b7c1-80a2aa594315 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:46.281593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 943/1887] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:54:46.333740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967f3b1-cdb4-4507-bfb2-8913c28de74b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:46.335477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0967f3b1-cdb4-4507-bfb2-8913c28de74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:46.348064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967f3b1-cdb4-4507-bfb2-8913c28de74b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:46.351784 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83384829-58e5-4f5b-96fb-44e3eefd90b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:46.352517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83384829-58e5-4f5b-96fb-44e3eefd90b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:46.352887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83384829-58e5-4f5b-96fb-44e3eefd90b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:46.353229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83384829-58e5-4f5b-96fb-44e3eefd90b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:46.413153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83384829-58e5-4f5b-96fb-44e3eefd90b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:46.414644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83384829-58e5-4f5b-96fb-44e3eefd90b4 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:46.424111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 945/1888] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:49.871010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8588d37-a2d9-44b6-b7c1-80a2aa594315 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:49.937781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8588d37-a2d9-44b6-b7c1-80a2aa594315 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:49.942801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:49.943717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:49.944002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:49.944277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] PATH_INFO: `/v3/users/7a8480ae60ca4e68a5258eb4cabe6348` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:49.945517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8480ae60ca4e68a5258eb4cabe6348)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:49.947766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:50.003961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8480ae60ca4e68a5258eb4cabe6348'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.014757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8480ae60ca4e68a5258eb4cabe6348'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.024646 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8480ae60ca4e68a5258eb4cabe6348'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.025490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8480ae60ca4e68a5258eb4cabe6348'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.026246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8480ae60ca4e68a5258eb4cabe6348'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.051176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 944/1889] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:49 2022] DELETE /identity/v3/users/7a8480ae60ca4e68a5258eb4cabe6348 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:50.053063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b7ce9-b958-4e36-8690-7b056a861532 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:50.053544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b7ce9-b958-4e36-8690-7b056a861532 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:50.053794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b7ce9-b958-4e36-8690-7b056a861532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:50.071223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-fb7b7ce9-b958-4e36-8690-7b056a861532 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:50.072526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-fb7b7ce9-b958-4e36-8690-7b056a861532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:50.096890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:50.138915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 946/1890] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:50.148925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:50.150671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:50.187661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:50.212990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a008b1-f272-4014-a86d-b9737647e880 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:50.216746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4b00e-da0d-4882-a045-913f36c5fc60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:50.217411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4b00e-da0d-4882-a045-913f36c5fc60 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:50.217758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4b00e-da0d-4882-a045-913f36c5fc60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:50.218032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4b00e-da0d-4882-a045-913f36c5fc60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:50.229002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4b00e-da0d-4882-a045-913f36c5fc60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:50.230437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4b00e-da0d-4882-a045-913f36c5fc60 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:50.247782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 945/1891] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:50.587068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b7ce9-b958-4e36-8690-7b056a861532 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:50.589058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-fb7b7ce9-b958-4e36-8690-7b056a861532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:50.602663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b7ce9-b958-4e36-8690-7b056a861532 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:50.623709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:50.624771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:50.625142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:50.625386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] PATH_INFO: `/v3/projects/45b26c38ff5444dd91998771eebfbb10` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:50.626786 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45b26c38ff5444dd91998771eebfbb10)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:50.629722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:50.697660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45b26c38ff5444dd91998771eebfbb10', 'request_id': 'req-ed2da903-f28e-4995-bc6b-a9bf12819c93'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.709424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45b26c38ff5444dd91998771eebfbb10', 'request_id': 'req-ed2da903-f28e-4995-bc6b-a9bf12819c93'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.717249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45b26c38ff5444dd91998771eebfbb10', 'request_id': 'req-ed2da903-f28e-4995-bc6b-a9bf12819c93'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.732415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] The token cache is being invalidate because project 45b26c38ff5444dd91998771eebfbb10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:50.732704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.733451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.738475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 947/1892] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:50 2022] DELETE /identity/v3/projects/45b26c38ff5444dd91998771eebfbb10 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:50.739544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4b00e-da0d-4882-a045-913f36c5fc60 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:50.750039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4b00e-da0d-4882-a045-913f36c5fc60 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:50.824458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:50.825359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:50.825642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:50.825914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] PATH_INFO: `/v3/users/7bb590bb03364f06b30bae47343a31cd` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:50.827096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bb590bb03364f06b30bae47343a31cd)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:50.829156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:50.882624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bb590bb03364f06b30bae47343a31cd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.893158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bb590bb03364f06b30bae47343a31cd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.902817 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bb590bb03364f06b30bae47343a31cd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.903776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bb590bb03364f06b30bae47343a31cd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.904613 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-980d9d27-3d61-484c-851e-cb886ebb61a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bb590bb03364f06b30bae47343a31cd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:50.927357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 946/1893] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:50 2022] DELETE /identity/v3/users/7bb590bb03364f06b30bae47343a31cd => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:54:51.325324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:51.327553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:51.342349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2da903-f28e-4995-bc6b-a9bf12819c93 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:51.401934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:51.402817 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:51.403091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:51.403393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] PATH_INFO: `/v3/projects/bbe4a412dbda4a0ea571f82d62f45d33` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:51.404563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbe4a412dbda4a0ea571f82d62f45d33)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:51.406646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:51.470185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe4a412dbda4a0ea571f82d62f45d33', 'request_id': 'req-93277ed4-4c9a-405d-9c61-c3943cc22d93'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:51.473190 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:51.495073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe4a412dbda4a0ea571f82d62f45d33', 'request_id': 'req-93277ed4-4c9a-405d-9c61-c3943cc22d93'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:51.527086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe4a412dbda4a0ea571f82d62f45d33', 'request_id': 'req-93277ed4-4c9a-405d-9c61-c3943cc22d93'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:51.546739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] The token cache is being invalidate because project bbe4a412dbda4a0ea571f82d62f45d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:51.547033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:51.547815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-93277ed4-4c9a-405d-9c61-c3943cc22d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:51.550008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 948/1894] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:51 2022] DELETE /identity/v3/projects/bbe4a412dbda4a0ea571f82d62f45d33 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:51.555066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a78c6-e0d2-412a-b1df-627ff6dca490 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:51.555409 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a78c6-e0d2-412a-b1df-627ff6dca490 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:51.555687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a78c6-e0d2-412a-b1df-627ff6dca490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:51.576906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d16a78c6-e0d2-412a-b1df-627ff6dca490 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:51.578286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d16a78c6-e0d2-412a-b1df-627ff6dca490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:51.656586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:51.672642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 947/1895] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:51.684399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46308e8b-5d11-41f0-aaf9-9c832ca2f261 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:51.684749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46308e8b-5d11-41f0-aaf9-9c832ca2f261 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:51.684993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46308e8b-5d11-41f0-aaf9-9c832ca2f261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:51.707189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-46308e8b-5d11-41f0-aaf9-9c832ca2f261 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:51.708459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-46308e8b-5d11-41f0-aaf9-9c832ca2f261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:51.717509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 949/1896] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:51.722248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a78c6-e0d2-412a-b1df-627ff6dca490 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:51.733372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a78c6-e0d2-412a-b1df-627ff6dca490 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:51.749176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4b618-5563-4189-87bf-81feefdcd125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:51.749969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4b618-5563-4189-87bf-81feefdcd125 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:51.750344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4b618-5563-4189-87bf-81feefdcd125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:51.750879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4b618-5563-4189-87bf-81feefdcd125 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:51.751808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4b618-5563-4189-87bf-81feefdcd125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:51.752348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4b618-5563-4189-87bf-81feefdcd125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:51.754220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4b618-5563-4189-87bf-81feefdcd125 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:51.771293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 948/1897] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:51.774743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46308e8b-5d11-41f0-aaf9-9c832ca2f261 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:51.784568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46308e8b-5d11-41f0-aaf9-9c832ca2f261 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:51.789624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1506f1-4de0-42cd-9668-3bd69b1d37cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:51.790320 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1506f1-4de0-42cd-9668-3bd69b1d37cf admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:51.790602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1506f1-4de0-42cd-9668-3bd69b1d37cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:51.790935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1506f1-4de0-42cd-9668-3bd69b1d37cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:51.791918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1506f1-4de0-42cd-9668-3bd69b1d37cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:51.793628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1506f1-4de0-42cd-9668-3bd69b1d37cf admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:51.823557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 950/1898] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:51 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:51.827493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4b618-5563-4189-87bf-81feefdcd125 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:51.838201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4b618-5563-4189-87bf-81feefdcd125 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:51.913932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905dbae-e2a3-496c-b9b4-8de7118f30cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:51.914935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905dbae-e2a3-496c-b9b4-8de7118f30cf admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:51.915365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905dbae-e2a3-496c-b9b4-8de7118f30cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:51.916104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905dbae-e2a3-496c-b9b4-8de7118f30cf admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:51.917200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905dbae-e2a3-496c-b9b4-8de7118f30cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:51.919302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905dbae-e2a3-496c-b9b4-8de7118f30cf admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:51.943726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 949/1899] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:51 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:51.948205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1506f1-4de0-42cd-9668-3bd69b1d37cf admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:51.960867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1506f1-4de0-42cd-9668-3bd69b1d37cf admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:51.966063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befca369-4a51-44f8-9d97-20829e01f190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:51.966858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befca369-4a51-44f8-9d97-20829e01f190 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:51.967438 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befca369-4a51-44f8-9d97-20829e01f190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:51.967902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befca369-4a51-44f8-9d97-20829e01f190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:51.968713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befca369-4a51-44f8-9d97-20829e01f190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:51.970645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befca369-4a51-44f8-9d97-20829e01f190 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:51.985709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 951/1900] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:51.990139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905dbae-e2a3-496c-b9b4-8de7118f30cf admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:52.002848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905dbae-e2a3-496c-b9b4-8de7118f30cf admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:52.010092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6b84e-0c29-49d9-bc80-400709f1052b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:52.010871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6b84e-0c29-49d9-bc80-400709f1052b admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.011198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6b84e-0c29-49d9-bc80-400709f1052b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.011599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6b84e-0c29-49d9-bc80-400709f1052b admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.012343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6b84e-0c29-49d9-bc80-400709f1052b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:52.015038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6b84e-0c29-49d9-bc80-400709f1052b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:52.032104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 950/1901] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:54:52.032771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befca369-4a51-44f8-9d97-20829e01f190 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:52.044080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befca369-4a51-44f8-9d97-20829e01f190 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:52.050788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2f169-a364-4d67-b77b-f6b1d3a7d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:52.051919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2f169-a364-4d67-b77b-f6b1d3a7d083 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.052315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2f169-a364-4d67-b77b-f6b1d3a7d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.053159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2f169-a364-4d67-b77b-f6b1d3a7d083 admin admin] PATH_INFO: `/v3/projects/482564ccc8854b4fb17c2995f93023fe/users/db16630c60254b48a19e6dbe234ed98c/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.070269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2f169-a364-4d67-b77b-f6b1d3a7d083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=482564ccc8854b4fb17c2995f93023fe, user_id=db16630c60254b48a19e6dbe234ed98c, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:52.073429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2f169-a364-4d67-b77b-f6b1d3a7d083 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:52.108360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 952/1902] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:52 2022] PUT /identity/v3/projects/482564ccc8854b4fb17c2995f93023fe/users/db16630c60254b48a19e6dbe234ed98c/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:54:52.113564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69e532-835f-4beb-aee7-9d009747b09d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.113882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69e532-835f-4beb-aee7-9d009747b09d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.114150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69e532-835f-4beb-aee7-9d009747b09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.156386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-5a69e532-835f-4beb-aee7-9d009747b09d None None] MFA Rules not processed for user `db16630c60254b48a19e6dbe234ed98c`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:52.157973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5a69e532-835f-4beb-aee7-9d009747b09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:52.291133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 951/1903] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:54:52.297128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2f169-a364-4d67-b77b-f6b1d3a7d083 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:52.364551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2f169-a364-4d67-b77b-f6b1d3a7d083 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:52.369451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64ed31-62fa-424f-9dbb-592710d473a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:52.370197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64ed31-62fa-424f-9dbb-592710d473a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.370463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64ed31-62fa-424f-9dbb-592710d473a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.370752 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64ed31-62fa-424f-9dbb-592710d473a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.371483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64ed31-62fa-424f-9dbb-592710d473a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:52.373449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64ed31-62fa-424f-9dbb-592710d473a6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:52.401541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 953/1904] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:52 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:52.405755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69e532-835f-4beb-aee7-9d009747b09d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:52.417127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69e532-835f-4beb-aee7-9d009747b09d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:52.473772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:52.485242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd2c06-4a4f-428a-8a6a-c9867d680a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:52.486087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd2c06-4a4f-428a-8a6a-c9867d680a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.486450 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd2c06-4a4f-428a-8a6a-c9867d680a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.486687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd2c06-4a4f-428a-8a6a-c9867d680a48 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.487543 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd2c06-4a4f-428a-8a6a-c9867d680a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:52.489587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd2c06-4a4f-428a-8a6a-c9867d680a48 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:52.516358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 952/1905] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:52 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:52.520556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64ed31-62fa-424f-9dbb-592710d473a6 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:52.531551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64ed31-62fa-424f-9dbb-592710d473a6 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:52.537360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c88aca-8330-4513-b208-851cfaf08340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:52.538044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c88aca-8330-4513-b208-851cfaf08340 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.538310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c88aca-8330-4513-b208-851cfaf08340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.538583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c88aca-8330-4513-b208-851cfaf08340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.539347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c88aca-8330-4513-b208-851cfaf08340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:52.540965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c88aca-8330-4513-b208-851cfaf08340 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:52.555397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 954/1906] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:52.558463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd2c06-4a4f-428a-8a6a-c9867d680a48 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:52.568812 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd2c06-4a4f-428a-8a6a-c9867d680a48 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:52.573438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29fd82-3bd0-473d-a925-6e53a3a1e5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:52.574413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29fd82-3bd0-473d-a925-6e53a3a1e5bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.574690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29fd82-3bd0-473d-a925-6e53a3a1e5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.574964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29fd82-3bd0-473d-a925-6e53a3a1e5bb admin admin] PATH_INFO: `/v3/projects/15c1f8c94cf2488dbd00fdae4a3b9b5a/users/69767e9c16384b1cbaab224e4cb92190/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.590370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29fd82-3bd0-473d-a925-6e53a3a1e5bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c1f8c94cf2488dbd00fdae4a3b9b5a, user_id=69767e9c16384b1cbaab224e4cb92190, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:52.593088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29fd82-3bd0-473d-a925-6e53a3a1e5bb admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:52.608004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 953/1907] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:52 2022] PUT /identity/v3/projects/15c1f8c94cf2488dbd00fdae4a3b9b5a/users/69767e9c16384b1cbaab224e4cb92190/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:52.612054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c88aca-8330-4513-b208-851cfaf08340 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:52.622367 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c88aca-8330-4513-b208-851cfaf08340 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:52.675145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:52.687841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac73b43-5ba0-42ba-aa24-5cd5050ad275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:52.688749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac73b43-5ba0-42ba-aa24-5cd5050ad275 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.689032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac73b43-5ba0-42ba-aa24-5cd5050ad275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.689306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac73b43-5ba0-42ba-aa24-5cd5050ad275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.690088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac73b43-5ba0-42ba-aa24-5cd5050ad275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:52.692674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac73b43-5ba0-42ba-aa24-5cd5050ad275 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:52.704411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 955/1908] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:52.708908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29fd82-3bd0-473d-a925-6e53a3a1e5bb admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:52.719206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29fd82-3bd0-473d-a925-6e53a3a1e5bb admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:52.723793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884fd31-9536-4e7f-be84-8157263e8b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:52.724690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884fd31-9536-4e7f-be84-8157263e8b74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.724971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884fd31-9536-4e7f-be84-8157263e8b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.725248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884fd31-9536-4e7f-be84-8157263e8b74 admin admin] PATH_INFO: `/v3/projects/15c1f8c94cf2488dbd00fdae4a3b9b5a/users/69767e9c16384b1cbaab224e4cb92190/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.727231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884fd31-9536-4e7f-be84-8157263e8b74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c1f8c94cf2488dbd00fdae4a3b9b5a, user_id=69767e9c16384b1cbaab224e4cb92190, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:52.729906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884fd31-9536-4e7f-be84-8157263e8b74 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:52.740890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 954/1909] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:52 2022] PUT /identity/v3/projects/15c1f8c94cf2488dbd00fdae4a3b9b5a/users/69767e9c16384b1cbaab224e4cb92190/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:52.751417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751d4d3-3abb-4e0b-b101-d330d1ba4348 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.751712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751d4d3-3abb-4e0b-b101-d330d1ba4348 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.752031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751d4d3-3abb-4e0b-b101-d330d1ba4348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.783811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-4751d4d3-3abb-4e0b-b101-d330d1ba4348 None None] MFA Rules not processed for user `69767e9c16384b1cbaab224e4cb92190`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:52.785486 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4751d4d3-3abb-4e0b-b101-d330d1ba4348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:52.858499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 956/1910] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:52.870082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884fd31-9536-4e7f-be84-8157263e8b74 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:52.871818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6884fd31-9536-4e7f-be84-8157263e8b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:52.937776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884fd31-9536-4e7f-be84-8157263e8b74 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:52.941503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299845a5-aeeb-4ab2-907e-f2f0ab0cc551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:52.942177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299845a5-aeeb-4ab2-907e-f2f0ab0cc551 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:52.942457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299845a5-aeeb-4ab2-907e-f2f0ab0cc551 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:52.942732 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299845a5-aeeb-4ab2-907e-f2f0ab0cc551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:52.952909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299845a5-aeeb-4ab2-907e-f2f0ab0cc551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:52.954562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299845a5-aeeb-4ab2-907e-f2f0ab0cc551 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:52.965359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 955/1911] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:53.032548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751d4d3-3abb-4e0b-b101-d330d1ba4348 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:53.097318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751d4d3-3abb-4e0b-b101-d330d1ba4348 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:53.102426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:53.103449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:53.103727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:53.104000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] PATH_INFO: `/v3/users/db16630c60254b48a19e6dbe234ed98c` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:53.105344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=db16630c60254b48a19e6dbe234ed98c)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:53.107447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:53.169244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db16630c60254b48a19e6dbe234ed98c'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:53.177640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db16630c60254b48a19e6dbe234ed98c'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:53.178365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db16630c60254b48a19e6dbe234ed98c'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:53.188806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db16630c60254b48a19e6dbe234ed98c'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:53.197820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:53.202825 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db16630c60254b48a19e6dbe234ed98c'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:53.210916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 957/1912] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:53 2022] DELETE /identity/v3/users/db16630c60254b48a19e6dbe234ed98c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:53.215095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f0ce8-89ff-4969-9b78-dbf88b827606 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:53.215490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f0ce8-89ff-4969-9b78-dbf88b827606 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:53.215763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f0ce8-89ff-4969-9b78-dbf88b827606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:53.236088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-166f0ce8-89ff-4969-9b78-dbf88b827606 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:53.237383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-166f0ce8-89ff-4969-9b78-dbf88b827606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:53.315467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 956/1913] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:53.351081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:53.353452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:53.423073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46555c22-1632-4b98-bd33-6e8a49028ada admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:53.427276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaf958-0870-407b-8a70-883c867988ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:53.427975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaf958-0870-407b-8a70-883c867988ce service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:53.428262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaf958-0870-407b-8a70-883c867988ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:53.428538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaf958-0870-407b-8a70-883c867988ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:53.444170 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaf958-0870-407b-8a70-883c867988ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:53.446162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaf958-0870-407b-8a70-883c867988ce service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:53.457135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 958/1914] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:54:53.877565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f0ce8-89ff-4969-9b78-dbf88b827606 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:53.879035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-166f0ce8-89ff-4969-9b78-dbf88b827606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:53.892065 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f0ce8-89ff-4969-9b78-dbf88b827606 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:53.907868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:53.908793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:53.909039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:53.909315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] PATH_INFO: `/v3/projects/482564ccc8854b4fb17c2995f93023fe` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:53.910615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=482564ccc8854b4fb17c2995f93023fe)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:53.912707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:53.963393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:53.973191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '482564ccc8854b4fb17c2995f93023fe', 'request_id': 'req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:53.983170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '482564ccc8854b4fb17c2995f93023fe', 'request_id': 'req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:53.990906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '482564ccc8854b4fb17c2995f93023fe', 'request_id': 'req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.006946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] The token cache is being invalidate because project 482564ccc8854b4fb17c2995f93023fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:54.007233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.007897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.012939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 957/1915] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:53 2022] DELETE /identity/v3/projects/482564ccc8854b4fb17c2995f93023fe => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:54.014119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaf958-0870-407b-8a70-883c867988ce service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:54.023989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaf958-0870-407b-8a70-883c867988ce service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:54.085733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:54.086464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:54.086785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:54.087003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] PATH_INFO: `/v3/users/69767e9c16384b1cbaab224e4cb92190` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:54.089786 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=69767e9c16384b1cbaab224e4cb92190)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:54.091754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:54.141053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69767e9c16384b1cbaab224e4cb92190'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.148394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69767e9c16384b1cbaab224e4cb92190'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.149107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69767e9c16384b1cbaab224e4cb92190'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.158879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69767e9c16384b1cbaab224e4cb92190'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.172584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bf582968-2c19-47aa-8650-eee64d2bf4ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69767e9c16384b1cbaab224e4cb92190'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.181102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 959/1916] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:54:54 2022] DELETE /identity/v3/users/69767e9c16384b1cbaab224e4cb92190 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:54:54.498759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:54.501690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:54.530757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75e897-0c96-4d24-b0bb-44e372d1e3e7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:54.592588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:54.593520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:54.593839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:54.594362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] PATH_INFO: `/v3/projects/15c1f8c94cf2488dbd00fdae4a3b9b5a` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:54.595721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c1f8c94cf2488dbd00fdae4a3b9b5a)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:54.597743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:54.668654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c1f8c94cf2488dbd00fdae4a3b9b5a', 'request_id': 'req-32f00cf5-9817-4ef1-809b-db69f0fa25f8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.678228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c1f8c94cf2488dbd00fdae4a3b9b5a', 'request_id': 'req-32f00cf5-9817-4ef1-809b-db69f0fa25f8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.686907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c1f8c94cf2488dbd00fdae4a3b9b5a', 'request_id': 'req-32f00cf5-9817-4ef1-809b-db69f0fa25f8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.738928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] The token cache is being invalidate because project 15c1f8c94cf2488dbd00fdae4a3b9b5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:54:54.739092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.745747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-32f00cf5-9817-4ef1-809b-db69f0fa25f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:54:54.760952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36b7d8-9ae0-4bfb-8d92-4ed36dd5e3af None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:54.761547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36b7d8-9ae0-4bfb-8d92-4ed36dd5e3af None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:54.761547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36b7d8-9ae0-4bfb-8d92-4ed36dd5e3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:54.763307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 958/1917] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:54:54 2022] DELETE /identity/v3/projects/15c1f8c94cf2488dbd00fdae4a3b9b5a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:54.799187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-de36b7d8-9ae0-4bfb-8d92-4ed36dd5e3af None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:54.805325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-de36b7d8-9ae0-4bfb-8d92-4ed36dd5e3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:54.816782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:55.018138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 960/1918] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:55.027018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea76ad-9f7c-43cf-89ba-016a9b62de14 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:55.027568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea76ad-9f7c-43cf-89ba-016a9b62de14 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:55.027568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea76ad-9f7c-43cf-89ba-016a9b62de14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:55.046288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2aea76ad-9f7c-43cf-89ba-016a9b62de14 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:55.047568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2aea76ad-9f7c-43cf-89ba-016a9b62de14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:55.056617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 959/1919] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:55.061457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36b7d8-9ae0-4bfb-8d92-4ed36dd5e3af None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:55.071815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36b7d8-9ae0-4bfb-8d92-4ed36dd5e3af None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:55.077253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53fe3a-4741-49e9-9ab9-cb400c8ef5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:55.077980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53fe3a-4741-49e9-9ab9-cb400c8ef5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:55.078380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53fe3a-4741-49e9-9ab9-cb400c8ef5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:55.078801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53fe3a-4741-49e9-9ab9-cb400c8ef5e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:55.079667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53fe3a-4741-49e9-9ab9-cb400c8ef5e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:54:55.079974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53fe3a-4741-49e9-9ab9-cb400c8ef5e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:55.081609 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53fe3a-4741-49e9-9ab9-cb400c8ef5e3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:55.095410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 961/1920] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:54:55.100082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea76ad-9f7c-43cf-89ba-016a9b62de14 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:55.109913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea76ad-9f7c-43cf-89ba-016a9b62de14 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:55.114721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952ec86-2af6-4dff-b52f-0921c44b494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:55.115442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952ec86-2af6-4dff-b52f-0921c44b494c admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:55.115720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952ec86-2af6-4dff-b52f-0921c44b494c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:55.116010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952ec86-2af6-4dff-b52f-0921c44b494c admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:55.116686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952ec86-2af6-4dff-b52f-0921c44b494c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:55.118611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952ec86-2af6-4dff-b52f-0921c44b494c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:55.148542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 960/1921] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:54:55 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:55.152891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53fe3a-4741-49e9-9ab9-cb400c8ef5e3 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:55.163502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53fe3a-4741-49e9-9ab9-cb400c8ef5e3 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:55.238062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d25cd-2076-4e7f-a968-f5d5e95e4bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:55.238748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d25cd-2076-4e7f-a968-f5d5e95e4bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:55.239031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d25cd-2076-4e7f-a968-f5d5e95e4bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:55.239327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d25cd-2076-4e7f-a968-f5d5e95e4bbe admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:55.240031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d25cd-2076-4e7f-a968-f5d5e95e4bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:55.242005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d25cd-2076-4e7f-a968-f5d5e95e4bbe admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:55.264810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 962/1922] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:54:55 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:54:55.268943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952ec86-2af6-4dff-b52f-0921c44b494c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:55.292453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952ec86-2af6-4dff-b52f-0921c44b494c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:55.303191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bfe7f-b8cc-4be3-ad9c-2f775b81e5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:55.303970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bfe7f-b8cc-4be3-ad9c-2f775b81e5bd admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:55.304256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bfe7f-b8cc-4be3-ad9c-2f775b81e5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:55.304554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bfe7f-b8cc-4be3-ad9c-2f775b81e5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:55.305346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bfe7f-b8cc-4be3-ad9c-2f775b81e5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:55.307240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bfe7f-b8cc-4be3-ad9c-2f775b81e5bd admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:55.323790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 961/1923] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:55.329308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d25cd-2076-4e7f-a968-f5d5e95e4bbe admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:55.341488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d25cd-2076-4e7f-a968-f5d5e95e4bbe admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:55.347079 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3c4f6-bdb4-4b5f-86a3-ba7ce62b0fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:55.347790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3c4f6-bdb4-4b5f-86a3-ba7ce62b0fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:55.348091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3c4f6-bdb4-4b5f-86a3-ba7ce62b0fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:55.348362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3c4f6-bdb4-4b5f-86a3-ba7ce62b0fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:55.349084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3c4f6-bdb4-4b5f-86a3-ba7ce62b0fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:55.350702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3c4f6-bdb4-4b5f-86a3-ba7ce62b0fd9 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:55.363297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 963/1924] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:54:55.365713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bfe7f-b8cc-4be3-ad9c-2f775b81e5bd admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:55.376773 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bfe7f-b8cc-4be3-ad9c-2f775b81e5bd admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:55.385179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb87753-cca4-4d50-9081-ead7731766b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:55.386284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb87753-cca4-4d50-9081-ead7731766b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:55.386712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb87753-cca4-4d50-9081-ead7731766b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:55.387158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb87753-cca4-4d50-9081-ead7731766b9 admin admin] PATH_INFO: `/v3/projects/866c19ba204540e9873b258ff6324e53/users/74bda29823a040aa94b9b8ffcab48f78/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:55.404733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb87753-cca4-4d50-9081-ead7731766b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=866c19ba204540e9873b258ff6324e53, user_id=74bda29823a040aa94b9b8ffcab48f78, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:55.407587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb87753-cca4-4d50-9081-ead7731766b9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:55.411129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:54:55.430926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 962/1925] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:54:55 2022] PUT /identity/v3/projects/866c19ba204540e9873b258ff6324e53/users/74bda29823a040aa94b9b8ffcab48f78/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:54:55.441749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b20761-11ac-46c9-939a-688ae88756aa None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:55.442071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b20761-11ac-46c9-939a-688ae88756aa None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:55.442294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b20761-11ac-46c9-939a-688ae88756aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:55.480920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-46b20761-11ac-46c9-939a-688ae88756aa None None] MFA Rules not processed for user `74bda29823a040aa94b9b8ffcab48f78`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:54:55.482178 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-46b20761-11ac-46c9-939a-688ae88756aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:55.613666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 964/1926] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:54:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:54:55.624591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb87753-cca4-4d50-9081-ead7731766b9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:54:55.627058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7bb87753-cca4-4d50-9081-ead7731766b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:54:55.707562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb87753-cca4-4d50-9081-ead7731766b9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:54:55.713326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dab7d-d01a-4e72-9a87-20361081fa5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:54:55.714501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dab7d-d01a-4e72-9a87-20361081fa5e service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:54:55.716822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dab7d-d01a-4e72-9a87-20361081fa5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:54:55.717275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dab7d-d01a-4e72-9a87-20361081fa5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:54:55.732209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dab7d-d01a-4e72-9a87-20361081fa5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:54:55.741157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dab7d-d01a-4e72-9a87-20361081fa5e service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:54:55.755603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 963/1927] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:54:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:05.604497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:05.749603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=28 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:15.280768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b20761-11ac-46c9-939a-688ae88756aa None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:15.346573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b20761-11ac-46c9-939a-688ae88756aa None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:15.351852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:15.352613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:15.352899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:15.353177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] PATH_INFO: `/v3/users/74bda29823a040aa94b9b8ffcab48f78` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:15.354382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74bda29823a040aa94b9b8ffcab48f78)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:15.356482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:15.472554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74bda29823a040aa94b9b8ffcab48f78'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:15.481995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74bda29823a040aa94b9b8ffcab48f78'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:15.483033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74bda29823a040aa94b9b8ffcab48f78'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:15.494525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74bda29823a040aa94b9b8ffcab48f78'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:15.518710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74bda29823a040aa94b9b8ffcab48f78'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:15.527722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 965/1928] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:15 2022] DELETE /identity/v3/users/74bda29823a040aa94b9b8ffcab48f78 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:15.541750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b5ba5-58d7-4783-94af-2ba3b6ad6437 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:15.542355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b5ba5-58d7-4783-94af-2ba3b6ad6437 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:15.543650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b5ba5-58d7-4783-94af-2ba3b6ad6437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:15.565213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-945b5ba5-58d7-4783-94af-2ba3b6ad6437 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:15.566698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-945b5ba5-58d7-4783-94af-2ba3b6ad6437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:15.643057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 964/1929] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:15.653911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:15.655509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:15.750151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86baad-a6e2-4c1c-a946-8dcefd17fa31 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:15.757205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c749fc-5967-4470-a2fe-183b9c1e925b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:15.757888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c749fc-5967-4470-a2fe-183b9c1e925b service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:15.758169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c749fc-5967-4470-a2fe-183b9c1e925b service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:15.758505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c749fc-5967-4470-a2fe-183b9c1e925b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:15.769471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c749fc-5967-4470-a2fe-183b9c1e925b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:15.771096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c749fc-5967-4470-a2fe-183b9c1e925b service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:15.788068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 966/1930] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:16.083267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b5ba5-58d7-4783-94af-2ba3b6ad6437 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:16.084833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-945b5ba5-58d7-4783-94af-2ba3b6ad6437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:16.106799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b5ba5-58d7-4783-94af-2ba3b6ad6437 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:16.112789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:16.113764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:16.114075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:16.114348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] PATH_INFO: `/v3/projects/866c19ba204540e9873b258ff6324e53` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:16.115912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=866c19ba204540e9873b258ff6324e53)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:16.117901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:16.178431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '866c19ba204540e9873b258ff6324e53', 'request_id': 'req-aa7ba054-0b24-47c1-840b-2a2d5ec39809'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:16.184905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '866c19ba204540e9873b258ff6324e53', 'request_id': 'req-aa7ba054-0b24-47c1-840b-2a2d5ec39809'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:16.190500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '866c19ba204540e9873b258ff6324e53', 'request_id': 'req-aa7ba054-0b24-47c1-840b-2a2d5ec39809'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:16.204971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] The token cache is being invalidate because project 866c19ba204540e9873b258ff6324e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:16.205253 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:16.205871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-aa7ba054-0b24-47c1-840b-2a2d5ec39809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:16.207710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 965/1931] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:16 2022] DELETE /identity/v3/projects/866c19ba204540e9873b258ff6324e53 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:17.424871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c749fc-5967-4470-a2fe-183b9c1e925b service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:17.426759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c9c749fc-5967-4470-a2fe-183b9c1e925b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:17.442941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c749fc-5967-4470-a2fe-183b9c1e925b service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:17.462275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:17.504949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:17.505853 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:17.506186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:17.506431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] PATH_INFO: `/v3/users/0a83557d4baf4280897c2e95a8937334` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:17.507815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a83557d4baf4280897c2e95a8937334)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:17.509782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:17.562081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a83557d4baf4280897c2e95a8937334'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:17.568888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a83557d4baf4280897c2e95a8937334'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:17.569551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a83557d4baf4280897c2e95a8937334'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:17.579154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a83557d4baf4280897c2e95a8937334'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:17.591151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a83557d4baf4280897c2e95a8937334'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:17.597061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 967/1932] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:17 2022] DELETE /identity/v3/users/0a83557d4baf4280897c2e95a8937334 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:17.601925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d398e-1817-4023-a318-46dd2cc12904 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:17.602528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d398e-1817-4023-a318-46dd2cc12904 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:17.602528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d398e-1817-4023-a318-46dd2cc12904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:17.620909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-be2d398e-1817-4023-a318-46dd2cc12904 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:17.622144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-be2d398e-1817-4023-a318-46dd2cc12904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:17.690029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 966/1933] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:17.701790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:17.704108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:17.768457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30baf9-c7a5-43eb-a20b-ef3e0138024d admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:17.772436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c80450-c85c-4add-8ab9-6ad549a7ff6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:17.773087 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c80450-c85c-4add-8ab9-6ad549a7ff6f service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:17.773366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c80450-c85c-4add-8ab9-6ad549a7ff6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:17.773638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c80450-c85c-4add-8ab9-6ad549a7ff6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:17.783463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c80450-c85c-4add-8ab9-6ad549a7ff6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:17.785588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c80450-c85c-4add-8ab9-6ad549a7ff6f service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:17.796905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 968/1934] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:18.097522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d398e-1817-4023-a318-46dd2cc12904 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:18.099355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-be2d398e-1817-4023-a318-46dd2cc12904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:18.115348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d398e-1817-4023-a318-46dd2cc12904 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:18.121292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:18.122349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:18.122595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:18.123740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] PATH_INFO: `/v3/projects/059c946aae6440d6b292327ae0a4a874` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:18.125696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=059c946aae6440d6b292327ae0a4a874)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:18.127686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:18.158569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:18.193348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '059c946aae6440d6b292327ae0a4a874', 'request_id': 'req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.201224 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '059c946aae6440d6b292327ae0a4a874', 'request_id': 'req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.210879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '059c946aae6440d6b292327ae0a4a874', 'request_id': 'req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.225888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] The token cache is being invalidate because project 059c946aae6440d6b292327ae0a4a874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:18.226176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.226788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.228397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 967/1935] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:18 2022] DELETE /identity/v3/projects/059c946aae6440d6b292327ae0a4a874 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:18.232562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c80450-c85c-4add-8ab9-6ad549a7ff6f service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:18.242304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c80450-c85c-4add-8ab9-6ad549a7ff6f service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:18.285041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:18.302173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:18.303156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:18.303462 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:18.303797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] PATH_INFO: `/v3/users/32a82424e7794dcd8ab6acb8bf605461` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:18.305003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a82424e7794dcd8ab6acb8bf605461)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:18.306941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:18.369152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a82424e7794dcd8ab6acb8bf605461'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.376327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a82424e7794dcd8ab6acb8bf605461'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.377022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a82424e7794dcd8ab6acb8bf605461'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.385929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a82424e7794dcd8ab6acb8bf605461'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.401094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-458ce755-9033-416a-8ec8-7af723dfc205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a82424e7794dcd8ab6acb8bf605461'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.410427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 969/1936] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:18 2022] DELETE /identity/v3/users/32a82424e7794dcd8ab6acb8bf605461 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:18.694913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:18.696530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:18.711502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447e285-cd41-489f-b7c8-0eb4caf9bfc6 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:18.777822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:18.778763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:18.779039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:18.779356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] PATH_INFO: `/v3/projects/7643802db0b143b3ab1d3acc7ffdffc6` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:18.780565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=7643802db0b143b3ab1d3acc7ffdffc6)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:18.782553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:18.831497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7643802db0b143b3ab1d3acc7ffdffc6', 'request_id': 'req-44871b4e-798d-4179-bb53-938e95bcc9ca'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.838010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7643802db0b143b3ab1d3acc7ffdffc6', 'request_id': 'req-44871b4e-798d-4179-bb53-938e95bcc9ca'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.845618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7643802db0b143b3ab1d3acc7ffdffc6', 'request_id': 'req-44871b4e-798d-4179-bb53-938e95bcc9ca'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.862289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] The token cache is being invalidate because project 7643802db0b143b3ab1d3acc7ffdffc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:18.862712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.863582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-44871b4e-798d-4179-bb53-938e95bcc9ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:18.869246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 968/1937] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:18 2022] DELETE /identity/v3/projects/7643802db0b143b3ab1d3acc7ffdffc6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:18.871729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528a748-3b7c-44a1-a310-61ea17043c7e None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:18.872289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528a748-3b7c-44a1-a310-61ea17043c7e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:18.872656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528a748-3b7c-44a1-a310-61ea17043c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:18.903657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-2528a748-3b7c-44a1-a310-61ea17043c7e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:18.904960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2528a748-3b7c-44a1-a310-61ea17043c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:18.970694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 970/1938] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:18.980396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a00024-a063-4764-87f2-3f6e91b0a528 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:18.980396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a00024-a063-4764-87f2-3f6e91b0a528 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:18.980396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a00024-a063-4764-87f2-3f6e91b0a528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:19.000158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-30a00024-a063-4764-87f2-3f6e91b0a528 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:19.001486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-30a00024-a063-4764-87f2-3f6e91b0a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:19.012729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 969/1939] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:19.013990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528a748-3b7c-44a1-a310-61ea17043c7e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:19.025552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528a748-3b7c-44a1-a310-61ea17043c7e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:19.032896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110054a-fdab-4594-ae97-b70bcc7a254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:19.033594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110054a-fdab-4594-ae97-b70bcc7a254c admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:19.033876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110054a-fdab-4594-ae97-b70bcc7a254c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:19.034243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110054a-fdab-4594-ae97-b70bcc7a254c admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:19.034993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110054a-fdab-4594-ae97-b70bcc7a254c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:55:19.035329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110054a-fdab-4594-ae97-b70bcc7a254c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:19.036997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110054a-fdab-4594-ae97-b70bcc7a254c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:19.056954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 971/1940] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:55:19.061373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a00024-a063-4764-87f2-3f6e91b0a528 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:19.071339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a00024-a063-4764-87f2-3f6e91b0a528 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:19.076459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cefce-d581-44f0-9cbc-17a295da101d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:19.077332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cefce-d581-44f0-9cbc-17a295da101d admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:19.077721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cefce-d581-44f0-9cbc-17a295da101d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:19.078097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cefce-d581-44f0-9cbc-17a295da101d admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:19.078906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cefce-d581-44f0-9cbc-17a295da101d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:19.081068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cefce-d581-44f0-9cbc-17a295da101d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:19.115638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 970/1941] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:19 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:55:19.117119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110054a-fdab-4594-ae97-b70bcc7a254c admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:19.128788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110054a-fdab-4594-ae97-b70bcc7a254c admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:19.185843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:19.199767 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1e9b8-03e9-4eb1-af43-3a79337fcad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:19.200593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e9b8-03e9-4eb1-af43-3a79337fcad7 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:19.200874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e9b8-03e9-4eb1-af43-3a79337fcad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:19.201146 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e9b8-03e9-4eb1-af43-3a79337fcad7 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:19.201889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1e9b8-03e9-4eb1-af43-3a79337fcad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:19.203951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1e9b8-03e9-4eb1-af43-3a79337fcad7 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:19.230692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 972/1942] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:19 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:19.234921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cefce-d581-44f0-9cbc-17a295da101d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:19.247078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cefce-d581-44f0-9cbc-17a295da101d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:19.252581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0384565-3db5-4dc3-8fce-ff4722ad8b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:19.253346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0384565-3db5-4dc3-8fce-ff4722ad8b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:19.253632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0384565-3db5-4dc3-8fce-ff4722ad8b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:19.253907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0384565-3db5-4dc3-8fce-ff4722ad8b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:19.254657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0384565-3db5-4dc3-8fce-ff4722ad8b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:19.256436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0384565-3db5-4dc3-8fce-ff4722ad8b39 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:19.259591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:19.272864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 971/1943] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:19.275865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1e9b8-03e9-4eb1-af43-3a79337fcad7 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:19.287697 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1e9b8-03e9-4eb1-af43-3a79337fcad7 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:19.292837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0da10-5d55-4878-a4a2-aee5cfb66731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:19.293662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0da10-5d55-4878-a4a2-aee5cfb66731 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:19.294068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0da10-5d55-4878-a4a2-aee5cfb66731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:19.294650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0da10-5d55-4878-a4a2-aee5cfb66731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:19.295417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0da10-5d55-4878-a4a2-aee5cfb66731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:19.297132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0da10-5d55-4878-a4a2-aee5cfb66731 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:19.309903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 973/1944] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:19.314882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0384565-3db5-4dc3-8fce-ff4722ad8b39 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:19.328129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0384565-3db5-4dc3-8fce-ff4722ad8b39 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:19.333118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069c693-9046-4b24-8adb-0974445276d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:19.334687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069c693-9046-4b24-8adb-0974445276d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:19.335022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069c693-9046-4b24-8adb-0974445276d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:19.335505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069c693-9046-4b24-8adb-0974445276d8 admin admin] PATH_INFO: `/v3/projects/d190902b5af14ce8a1ff1710687fb6ce/users/dd9f942307f2405186d79f6e2a0acdf7/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:19.352463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069c693-9046-4b24-8adb-0974445276d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d190902b5af14ce8a1ff1710687fb6ce, user_id=dd9f942307f2405186d79f6e2a0acdf7, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:19.355206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069c693-9046-4b24-8adb-0974445276d8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:19.369965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 972/1945] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:19 2022] PUT /identity/v3/projects/d190902b5af14ce8a1ff1710687fb6ce/users/dd9f942307f2405186d79f6e2a0acdf7/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:19.375203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa8ff-88df-4214-841c-e09a2323b811 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:19.375605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa8ff-88df-4214-841c-e09a2323b811 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:19.375882 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa8ff-88df-4214-841c-e09a2323b811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:19.394240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-8f3aa8ff-88df-4214-841c-e09a2323b811 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:19.395521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8f3aa8ff-88df-4214-841c-e09a2323b811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:19.471878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 974/1946] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:19.482753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069c693-9046-4b24-8adb-0974445276d8 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:19.484638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c069c693-9046-4b24-8adb-0974445276d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:19.565481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069c693-9046-4b24-8adb-0974445276d8 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:19.571559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b689e1-a118-41d8-86bb-08584558eec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:19.572330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b689e1-a118-41d8-86bb-08584558eec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:19.572687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b689e1-a118-41d8-86bb-08584558eec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:19.573044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b689e1-a118-41d8-86bb-08584558eec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:19.584644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b689e1-a118-41d8-86bb-08584558eec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:19.586779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b689e1-a118-41d8-86bb-08584558eec6 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:19.602801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 973/1947] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:20.484661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd1b15-cad0-4336-a5ed-c8e32e447d50 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:20.485225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd1b15-cad0-4336-a5ed-c8e32e447d50 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:20.485225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd1b15-cad0-4336-a5ed-c8e32e447d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:20.518549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-97dd1b15-cad0-4336-a5ed-c8e32e447d50 None None] MFA Rules not processed for user `dd9f942307f2405186d79f6e2a0acdf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:20.520012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-97dd1b15-cad0-4336-a5ed-c8e32e447d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:20.596823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 975/1948] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:20.601935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b689e1-a118-41d8-86bb-08584558eec6 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:20.612586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b689e1-a118-41d8-86bb-08584558eec6 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:20.617550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f78d57-ab68-462d-99fa-8de73af83fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:20.618241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f78d57-ab68-462d-99fa-8de73af83fef admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:20.618522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f78d57-ab68-462d-99fa-8de73af83fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:20.618795 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f78d57-ab68-462d-99fa-8de73af83fef admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:20.619566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f78d57-ab68-462d-99fa-8de73af83fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:20.621501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f78d57-ab68-462d-99fa-8de73af83fef admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:20.653232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 974/1949] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:20 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:20.654466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd1b15-cad0-4336-a5ed-c8e32e447d50 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:20.668329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd1b15-cad0-4336-a5ed-c8e32e447d50 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:20.734268 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63835025-6820-446d-97b7-66faa0c092b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:20.734973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835025-6820-446d-97b7-66faa0c092b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:20.735321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835025-6820-446d-97b7-66faa0c092b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:20.735623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835025-6820-446d-97b7-66faa0c092b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:20.736364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63835025-6820-446d-97b7-66faa0c092b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:20.738416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63835025-6820-446d-97b7-66faa0c092b4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:20.746416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:20.765088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 976/1950] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:20 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:20.768966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f78d57-ab68-462d-99fa-8de73af83fef admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:20.779823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f78d57-ab68-462d-99fa-8de73af83fef admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:20.784402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec77cbd-8d91-4d7f-b5d8-76321452541b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:20.785082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec77cbd-8d91-4d7f-b5d8-76321452541b admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:20.785354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec77cbd-8d91-4d7f-b5d8-76321452541b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:20.785626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec77cbd-8d91-4d7f-b5d8-76321452541b admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:20.786350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec77cbd-8d91-4d7f-b5d8-76321452541b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:20.788019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec77cbd-8d91-4d7f-b5d8-76321452541b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:20.807553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 975/1951] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:20.811399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63835025-6820-446d-97b7-66faa0c092b4 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:20.823305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63835025-6820-446d-97b7-66faa0c092b4 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:20.831929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8693785-0f72-4b52-987e-7a8b6f46c6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:20.832989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8693785-0f72-4b52-987e-7a8b6f46c6d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:20.833285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8693785-0f72-4b52-987e-7a8b6f46c6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:20.833548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8693785-0f72-4b52-987e-7a8b6f46c6d2 admin admin] PATH_INFO: `/v3/projects/7b6fb918858246c8bd4871ea6e867480/users/7b3a3a277aa343a5ab4f24c8ee6849af/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:20.850505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8693785-0f72-4b52-987e-7a8b6f46c6d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6fb918858246c8bd4871ea6e867480, user_id=7b3a3a277aa343a5ab4f24c8ee6849af, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:20.853321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8693785-0f72-4b52-987e-7a8b6f46c6d2 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:20.872556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 977/1952] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:20 2022] PUT /identity/v3/projects/7b6fb918858246c8bd4871ea6e867480/users/7b3a3a277aa343a5ab4f24c8ee6849af/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:20.876507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec77cbd-8d91-4d7f-b5d8-76321452541b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:20.888622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec77cbd-8d91-4d7f-b5d8-76321452541b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:20.977198 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa0e4b-0494-47bb-bc33-5e7239ffb5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:20.977889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0e4b-0494-47bb-bc33-5e7239ffb5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:20.978335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0e4b-0494-47bb-bc33-5e7239ffb5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:20.978450 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0e4b-0494-47bb-bc33-5e7239ffb5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:20.979292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa0e4b-0494-47bb-bc33-5e7239ffb5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:20.981154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa0e4b-0494-47bb-bc33-5e7239ffb5e1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:20.995603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 976/1953] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:20.996841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8693785-0f72-4b52-987e-7a8b6f46c6d2 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:21.013135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8693785-0f72-4b52-987e-7a8b6f46c6d2 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:21.018525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361d556-8d71-448a-84f6-6d57a6249ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:21.019559 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361d556-8d71-448a-84f6-6d57a6249ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:21.019908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361d556-8d71-448a-84f6-6d57a6249ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:21.020375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361d556-8d71-448a-84f6-6d57a6249ec7 admin admin] PATH_INFO: `/v3/projects/7b6fb918858246c8bd4871ea6e867480/users/7b3a3a277aa343a5ab4f24c8ee6849af/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:21.022728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361d556-8d71-448a-84f6-6d57a6249ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6fb918858246c8bd4871ea6e867480, user_id=7b3a3a277aa343a5ab4f24c8ee6849af, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:21.025606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361d556-8d71-448a-84f6-6d57a6249ec7 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:21.041461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 978/1954] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:20 2022] PUT /identity/v3/projects/7b6fb918858246c8bd4871ea6e867480/users/7b3a3a277aa343a5ab4f24c8ee6849af/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:21.678581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e548485-c116-45b1-9a86-30a9add8c9a0 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:21.679081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e548485-c116-45b1-9a86-30a9add8c9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:21.679201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e548485-c116-45b1-9a86-30a9add8c9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:21.707206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2e548485-c116-45b1-9a86-30a9add8c9a0 None None] MFA Rules not processed for user `7b3a3a277aa343a5ab4f24c8ee6849af`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:21.708417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2e548485-c116-45b1-9a86-30a9add8c9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:21.742457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:21.791286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 977/1955] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:21.803214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361d556-8d71-448a-84f6-6d57a6249ec7 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:21.805105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5361d556-8d71-448a-84f6-6d57a6249ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:21.865354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361d556-8d71-448a-84f6-6d57a6249ec7 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:21.870285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a78fdf-98f3-415b-a8cc-fb830588ff84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:21.870911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a78fdf-98f3-415b-a8cc-fb830588ff84 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:21.871189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a78fdf-98f3-415b-a8cc-fb830588ff84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:21.871499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a78fdf-98f3-415b-a8cc-fb830588ff84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:21.933431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a78fdf-98f3-415b-a8cc-fb830588ff84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:21.934887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a78fdf-98f3-415b-a8cc-fb830588ff84 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:21.943908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 979/1956] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:24.378762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e548485-c116-45b1-9a86-30a9add8c9a0 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:24.381512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2e548485-c116-45b1-9a86-30a9add8c9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:24.401400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e548485-c116-45b1-9a86-30a9add8c9a0 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:24.406491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bc548-3caf-4ab2-81d8-e58ff6d5268f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:24.407519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bc548-3caf-4ab2-81d8-e58ff6d5268f service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:24.407922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bc548-3caf-4ab2-81d8-e58ff6d5268f service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:24.408219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bc548-3caf-4ab2-81d8-e58ff6d5268f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:24.422063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bc548-3caf-4ab2-81d8-e58ff6d5268f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:24.424297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bc548-3caf-4ab2-81d8-e58ff6d5268f service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:24.439385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 978/1957] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:55:27.477478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a78fdf-98f3-415b-a8cc-fb830588ff84 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:27.478987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-59a78fdf-98f3-415b-a8cc-fb830588ff84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:27.552012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a78fdf-98f3-415b-a8cc-fb830588ff84 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:27.556730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:27.557608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:27.557897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:27.558168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] PATH_INFO: `/v3/users/3e7874ec8ade4efcb67006400ac916d5` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:27.559458 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7874ec8ade4efcb67006400ac916d5)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:27.561461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:27.567307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:27.615002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7874ec8ade4efcb67006400ac916d5'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:27.624840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7874ec8ade4efcb67006400ac916d5'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:27.625579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7874ec8ade4efcb67006400ac916d5'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:27.636245 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7874ec8ade4efcb67006400ac916d5'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:27.650002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7874ec8ade4efcb67006400ac916d5'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:27.656371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 980/1958] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:27 2022] DELETE /identity/v3/users/3e7874ec8ade4efcb67006400ac916d5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:27.661127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df930a1e-11f5-4d28-8041-ff00f56c1139 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:27.661601 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df930a1e-11f5-4d28-8041-ff00f56c1139 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:27.661696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df930a1e-11f5-4d28-8041-ff00f56c1139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:27.685527 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-df930a1e-11f5-4d28-8041-ff00f56c1139 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:27.687231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-df930a1e-11f5-4d28-8041-ff00f56c1139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:27.775292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 979/1959] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:27.786015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:27.787668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:27.855447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46455c-a76f-447a-8662-1391fdbe863c admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:27.859826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c505da7-64b5-4c94-b036-0b2f6ca8bda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:27.860487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c505da7-64b5-4c94-b036-0b2f6ca8bda3 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:27.860774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c505da7-64b5-4c94-b036-0b2f6ca8bda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:27.861037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c505da7-64b5-4c94-b036-0b2f6ca8bda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:27.875639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c505da7-64b5-4c94-b036-0b2f6ca8bda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:27.877219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c505da7-64b5-4c94-b036-0b2f6ca8bda3 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:27.892622 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 981/1960] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:28.545284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df930a1e-11f5-4d28-8041-ff00f56c1139 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:28.546751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-df930a1e-11f5-4d28-8041-ff00f56c1139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:28.573513 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df930a1e-11f5-4d28-8041-ff00f56c1139 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:28.585664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:28.586791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:28.587195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:28.587747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] PATH_INFO: `/v3/projects/7705e68009b542cc97318cd75880ecd6` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:28.589218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7705e68009b542cc97318cd75880ecd6)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:28.591481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:28.675704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7705e68009b542cc97318cd75880ecd6', 'request_id': 'req-4ae43c60-4912-4fbf-afcd-12715de70ed7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:28.683079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7705e68009b542cc97318cd75880ecd6', 'request_id': 'req-4ae43c60-4912-4fbf-afcd-12715de70ed7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:28.690189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7705e68009b542cc97318cd75880ecd6', 'request_id': 'req-4ae43c60-4912-4fbf-afcd-12715de70ed7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:28.706237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] The token cache is being invalidate because project 7705e68009b542cc97318cd75880ecd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:28.706588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:28.707348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:28.709607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 980/1961] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:28 2022] DELETE /identity/v3/projects/7705e68009b542cc97318cd75880ecd6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:28.714439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c505da7-64b5-4c94-b036-0b2f6ca8bda3 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:28.727625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c505da7-64b5-4c94-b036-0b2f6ca8bda3 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:28.805070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:28.806639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:28.807051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:28.807348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] PATH_INFO: `/v3/users/a737b075e35b43d5ac93aea960ab2664` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:28.808763 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a737b075e35b43d5ac93aea960ab2664)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:28.810907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:28.899038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:28.917395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a737b075e35b43d5ac93aea960ab2664'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:28.929372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a737b075e35b43d5ac93aea960ab2664'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:28.931471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a737b075e35b43d5ac93aea960ab2664'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:28.951044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a737b075e35b43d5ac93aea960ab2664'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:28.972256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a737b075e35b43d5ac93aea960ab2664'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:28.980561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 982/1962] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:28 2022] DELETE /identity/v3/users/a737b075e35b43d5ac93aea960ab2664 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:29.068738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.070297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:29.084298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae43c60-4912-4fbf-afcd-12715de70ed7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.118308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:29.153215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.154129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.154415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.154714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] PATH_INFO: `/v3/users/3e05d19530a74e05a11be774102b425a` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.156060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e05d19530a74e05a11be774102b425a)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.160612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.209511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e05d19530a74e05a11be774102b425a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.216691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e05d19530a74e05a11be774102b425a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.227229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e05d19530a74e05a11be774102b425a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.227982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e05d19530a74e05a11be774102b425a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.229084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e05d19530a74e05a11be774102b425a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.250977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 981/1963] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:29 2022] DELETE /identity/v3/users/3e05d19530a74e05a11be774102b425a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:29.276092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.277795 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:29.295572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8c834-0735-4f3b-bdd4-5d74cbb54638 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.356732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.357753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.358099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.358495 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] PATH_INFO: `/v3/projects/b9fcbf7d7fa94b679eae037650dba255` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.359790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9fcbf7d7fa94b679eae037650dba255)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.361787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.417340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9fcbf7d7fa94b679eae037650dba255', 'request_id': 'req-1a095902-1f57-4b37-9182-0adde40cc683'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.424712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9fcbf7d7fa94b679eae037650dba255', 'request_id': 'req-1a095902-1f57-4b37-9182-0adde40cc683'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.430906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9fcbf7d7fa94b679eae037650dba255', 'request_id': 'req-1a095902-1f57-4b37-9182-0adde40cc683'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.437664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.439522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:29.450539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] The token cache is being invalidate because project b9fcbf7d7fa94b679eae037650dba255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:29.450819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.451679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1a095902-1f57-4b37-9182-0adde40cc683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.453386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 983/1964] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:29 2022] DELETE /identity/v3/projects/b9fcbf7d7fa94b679eae037650dba255 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:29.453950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb53da-b68d-4687-a767-9722e4a77283 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.458330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12095034-ce39-4296-a597-1f3c3055d491 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.458642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12095034-ce39-4296-a597-1f3c3055d491 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.458915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12095034-ce39-4296-a597-1f3c3055d491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.477886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-12095034-ce39-4296-a597-1f3c3055d491 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:29.479088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-12095034-ce39-4296-a597-1f3c3055d491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:29.522351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:29.535542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.537260 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.537375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.537588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] PATH_INFO: `/v3/projects/413717d499504cb29cb26d08bedf7e0a` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.538802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] RBAC: Authorizing `identity:delete_project(project_id=413717d499504cb29cb26d08bedf7e0a)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.541026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.552603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 984/1965] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:29.560812 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d3641-d6a0-41fd-8305-0687fc66492b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.561136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d3641-d6a0-41fd-8305-0687fc66492b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.561375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d3641-d6a0-41fd-8305-0687fc66492b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.579795 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-1b9d3641-d6a0-41fd-8305-0687fc66492b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:29.581001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1b9d3641-d6a0-41fd-8305-0687fc66492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:29.604472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '413717d499504cb29cb26d08bedf7e0a', 'request_id': 'req-4269f456-124b-401e-8b7b-a472873d734a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.610899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '413717d499504cb29cb26d08bedf7e0a', 'request_id': 'req-4269f456-124b-401e-8b7b-a472873d734a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.612890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:29.618011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '413717d499504cb29cb26d08bedf7e0a', 'request_id': 'req-4269f456-124b-401e-8b7b-a472873d734a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.634549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] The token cache is being invalidate because project 413717d499504cb29cb26d08bedf7e0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:29.634858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.635666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4269f456-124b-401e-8b7b-a472873d734a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:29.637337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 982/1966] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:29 2022] DELETE /identity/v3/projects/413717d499504cb29cb26d08bedf7e0a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:29.642436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010124b8-f7d9-44d9-89e6-832e626dead9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.642725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010124b8-f7d9-44d9-89e6-832e626dead9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.642990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010124b8-f7d9-44d9-89e6-832e626dead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.650608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 985/1967] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:29.655109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d3641-d6a0-41fd-8305-0687fc66492b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.662490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-010124b8-f7d9-44d9-89e6-832e626dead9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:29.663749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-010124b8-f7d9-44d9-89e6-832e626dead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:29.664162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d3641-d6a0-41fd-8305-0687fc66492b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.668939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4be57-ab8b-45d0-b261-4e24ce385a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.669606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4be57-ab8b-45d0-b261-4e24ce385a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.669888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4be57-ab8b-45d0-b261-4e24ce385a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.670161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4be57-ab8b-45d0-b261-4e24ce385a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.670897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4be57-ab8b-45d0-b261-4e24ce385a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:55:29.671212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4be57-ab8b-45d0-b261-4e24ce385a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.672212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 983/1968] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:29.679812 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4be57-ab8b-45d0-b261-4e24ce385a67 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.681256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6fb9-381b-4939-bc79-d801858fd23f None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.681736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6fb9-381b-4939-bc79-d801858fd23f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.681853 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6fb9-381b-4939-bc79-d801858fd23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.685639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 986/1969] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:55:29.689703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4be57-ab8b-45d0-b261-4e24ce385a67 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.699216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4be57-ab8b-45d0-b261-4e24ce385a67 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.703879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d788b9-1c7d-4d88-a53a-a6c5e5ed8f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.704106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-24ca6fb9-381b-4939-bc79-d801858fd23f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:29.704520 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d788b9-1c7d-4d88-a53a-a6c5e5ed8f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.704804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d788b9-1c7d-4d88-a53a-a6c5e5ed8f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.705081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d788b9-1c7d-4d88-a53a-a6c5e5ed8f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.705333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-24ca6fb9-381b-4939-bc79-d801858fd23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:29.705762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d788b9-1c7d-4d88-a53a-a6c5e5ed8f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.707705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d788b9-1c7d-4d88-a53a-a6c5e5ed8f55 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.714295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 984/1970] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:29.718601 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca6fb9-381b-4939-bc79-d801858fd23f None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.728028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca6fb9-381b-4939-bc79-d801858fd23f None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.733509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367c754-205a-4f10-9aaa-7f27ff2ff904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.734180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367c754-205a-4f10-9aaa-7f27ff2ff904 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.734469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367c754-205a-4f10-9aaa-7f27ff2ff904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.738059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 987/1971] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:29 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:29.738472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367c754-205a-4f10-9aaa-7f27ff2ff904 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.738472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367c754-205a-4f10-9aaa-7f27ff2ff904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:55:29.738472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367c754-205a-4f10-9aaa-7f27ff2ff904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.738472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367c754-205a-4f10-9aaa-7f27ff2ff904 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.739032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d788b9-1c7d-4d88-a53a-a6c5e5ed8f55 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.750027 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 985/1972] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:55:29.754183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367c754-205a-4f10-9aaa-7f27ff2ff904 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.764052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367c754-205a-4f10-9aaa-7f27ff2ff904 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.808548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d788b9-1c7d-4d88-a53a-a6c5e5ed8f55 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.813193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d323636-08d0-4010-a561-e9afafbf2ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.813870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d323636-08d0-4010-a561-e9afafbf2ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.814146 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d323636-08d0-4010-a561-e9afafbf2ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.814415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d323636-08d0-4010-a561-e9afafbf2ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.815131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d323636-08d0-4010-a561-e9afafbf2ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.817082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d323636-08d0-4010-a561-e9afafbf2ed5 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.823338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b0a5c-6dbd-46a6-92c2-4506c6d3153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.824047 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b0a5c-6dbd-46a6-92c2-4506c6d3153e admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.824330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b0a5c-6dbd-46a6-92c2-4506c6d3153e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.824638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b0a5c-6dbd-46a6-92c2-4506c6d3153e admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.825336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b0a5c-6dbd-46a6-92c2-4506c6d3153e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.827397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b0a5c-6dbd-46a6-92c2-4506c6d3153e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.838730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 988/1973] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:29 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:55:29.843045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d323636-08d0-4010-a561-e9afafbf2ed5 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.853009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d323636-08d0-4010-a561-e9afafbf2ed5 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.855728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 986/1974] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:29 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:55:29.857902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb79c9-8ffd-4e15-aeb0-b7dc59c2ecb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.858567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb79c9-8ffd-4e15-aeb0-b7dc59c2ecb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.858941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb79c9-8ffd-4e15-aeb0-b7dc59c2ecb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.859210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb79c9-8ffd-4e15-aeb0-b7dc59c2ecb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.859868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b0a5c-6dbd-46a6-92c2-4506c6d3153e admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.860020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb79c9-8ffd-4e15-aeb0-b7dc59c2ecb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.861521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb79c9-8ffd-4e15-aeb0-b7dc59c2ecb3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.872147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 989/1975] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:29.875662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb79c9-8ffd-4e15-aeb0-b7dc59c2ecb3 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.885059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb79c9-8ffd-4e15-aeb0-b7dc59c2ecb3 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.905828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:29.958393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea351a-93ef-423e-a73b-eebae3548111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.960246 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea351a-93ef-423e-a73b-eebae3548111 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.961051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea351a-93ef-423e-a73b-eebae3548111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.961334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea351a-93ef-423e-a73b-eebae3548111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.962088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea351a-93ef-423e-a73b-eebae3548111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.963800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea351a-93ef-423e-a73b-eebae3548111 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.967062 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b0a5c-6dbd-46a6-92c2-4506c6d3153e admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:29.974338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f174aa6-d186-4882-bfe7-42af9c422033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:29.975365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f174aa6-d186-4882-bfe7-42af9c422033 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:29.975725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f174aa6-d186-4882-bfe7-42af9c422033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:29.976802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f174aa6-d186-4882-bfe7-42af9c422033 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:29.976952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 990/1976] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:55:29.977965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f174aa6-d186-4882-bfe7-42af9c422033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:29.980182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f174aa6-d186-4882-bfe7-42af9c422033 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:29.981523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea351a-93ef-423e-a73b-eebae3548111 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:29.996855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea351a-93ef-423e-a73b-eebae3548111 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.003606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe1abc-f8fa-4cd0-8dcf-0d7ae2cd9cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.004634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe1abc-f8fa-4cd0-8dcf-0d7ae2cd9cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.004946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe1abc-f8fa-4cd0-8dcf-0d7ae2cd9cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.005223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe1abc-f8fa-4cd0-8dcf-0d7ae2cd9cd4 admin admin] PATH_INFO: `/v3/projects/5c7148b192884425a79290bcba1beee0/users/41a29a98d5fe48b3ab4501b86aef6061/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.010382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 987/1977] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:29 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:55:30.014850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f174aa6-d186-4882-bfe7-42af9c422033 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.025268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f174aa6-d186-4882-bfe7-42af9c422033 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.025650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe1abc-f8fa-4cd0-8dcf-0d7ae2cd9cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c7148b192884425a79290bcba1beee0, user_id=41a29a98d5fe48b3ab4501b86aef6061, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.028531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe1abc-f8fa-4cd0-8dcf-0d7ae2cd9cd4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.030231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086d42a-8ccb-4a92-b22e-f94cea3ed817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.030910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086d42a-8ccb-4a92-b22e-f94cea3ed817 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.031196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086d42a-8ccb-4a92-b22e-f94cea3ed817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.031508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086d42a-8ccb-4a92-b22e-f94cea3ed817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.032622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086d42a-8ccb-4a92-b22e-f94cea3ed817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.034245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086d42a-8ccb-4a92-b22e-f94cea3ed817 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.046721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 991/1978] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:29 2022] PUT /identity/v3/projects/5c7148b192884425a79290bcba1beee0/users/41a29a98d5fe48b3ab4501b86aef6061/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:30.047596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 988/1979] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:30.052309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe1abc-f8fa-4cd0-8dcf-0d7ae2cd9cd4 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.058290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4521a-859a-4f7e-99e0-60b9122d9aa8 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.058642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4521a-859a-4f7e-99e0-60b9122d9aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.059063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4521a-859a-4f7e-99e0-60b9122d9aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.091336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-1cd4521a-859a-4f7e-99e0-60b9122d9aa8 None None] MFA Rules not processed for user `41a29a98d5fe48b3ab4501b86aef6061`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:30.093077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1cd4521a-859a-4f7e-99e0-60b9122d9aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:30.102412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:30.127759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe1abc-f8fa-4cd0-8dcf-0d7ae2cd9cd4 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.134017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db0d81-65b6-407c-9c2d-feb7292c8ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.135276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db0d81-65b6-407c-9c2d-feb7292c8ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.135748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db0d81-65b6-407c-9c2d-feb7292c8ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.136213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db0d81-65b6-407c-9c2d-feb7292c8ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.137132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db0d81-65b6-407c-9c2d-feb7292c8ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.139859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db0d81-65b6-407c-9c2d-feb7292c8ed2 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.151611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 992/1980] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:30.156296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db0d81-65b6-407c-9c2d-feb7292c8ed2 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.166655 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db0d81-65b6-407c-9c2d-feb7292c8ed2 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.171819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3ccaf-df31-447a-8293-4ecf9a30bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.172716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3ccaf-df31-447a-8293-4ecf9a30bd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.172996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3ccaf-df31-447a-8293-4ecf9a30bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.173610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3ccaf-df31-447a-8293-4ecf9a30bd41 admin admin] PATH_INFO: `/v3/projects/27aa4fc6d2bd46d3a3492cf0b1d7ae9d/users/93ae888c6e364e2e96ff52e63415a0e5/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.177142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 989/1981] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:30.181749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4521a-859a-4f7e-99e0-60b9122d9aa8 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.192787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3ccaf-df31-447a-8293-4ecf9a30bd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27aa4fc6d2bd46d3a3492cf0b1d7ae9d, user_id=93ae888c6e364e2e96ff52e63415a0e5, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.194398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4521a-859a-4f7e-99e0-60b9122d9aa8 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.196004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3ccaf-df31-447a-8293-4ecf9a30bd41 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.199184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7d8f0-cc7a-492a-b119-e789c9bd37c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.199952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7d8f0-cc7a-492a-b119-e789c9bd37c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.200242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7d8f0-cc7a-492a-b119-e789c9bd37c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.200519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7d8f0-cc7a-492a-b119-e789c9bd37c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.201317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7d8f0-cc7a-492a-b119-e789c9bd37c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.203352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7d8f0-cc7a-492a-b119-e789c9bd37c5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.220597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 993/1982] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:30 2022] PUT /identity/v3/projects/27aa4fc6d2bd46d3a3492cf0b1d7ae9d/users/93ae888c6e364e2e96ff52e63415a0e5/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:30.225964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b94bbc-a303-4d34-a1cf-f47696bcd928 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.226252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b94bbc-a303-4d34-a1cf-f47696bcd928 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.226525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b94bbc-a303-4d34-a1cf-f47696bcd928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.234385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 990/1983] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:30 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:55:30.235393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7d8f0-cc7a-492a-b119-e789c9bd37c5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.243702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c3b94bbc-a303-4d34-a1cf-f47696bcd928 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:30.244967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c3b94bbc-a303-4d34-a1cf-f47696bcd928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:30.299890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7d8f0-cc7a-492a-b119-e789c9bd37c5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.305216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066de76-60eb-4ec8-8f34-3a8e9263ad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.305814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 994/1984] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:30.306089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066de76-60eb-4ec8-8f34-3a8e9263ad2d admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.306349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066de76-60eb-4ec8-8f34-3a8e9263ad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.306700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066de76-60eb-4ec8-8f34-3a8e9263ad2d admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.307435 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066de76-60eb-4ec8-8f34-3a8e9263ad2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.309977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066de76-60eb-4ec8-8f34-3a8e9263ad2d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.315856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b94bbc-a303-4d34-a1cf-f47696bcd928 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.317504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c3b94bbc-a303-4d34-a1cf-f47696bcd928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:30.328275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:30.331017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 991/1985] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:30 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:30.335268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066de76-60eb-4ec8-8f34-3a8e9263ad2d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.347417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066de76-60eb-4ec8-8f34-3a8e9263ad2d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.352185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ca4b7-b8eb-43fa-ab9c-9e5dab7a37d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.352849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ca4b7-b8eb-43fa-ab9c-9e5dab7a37d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.353130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ca4b7-b8eb-43fa-ab9c-9e5dab7a37d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.353396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ca4b7-b8eb-43fa-ab9c-9e5dab7a37d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.354145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ca4b7-b8eb-43fa-ab9c-9e5dab7a37d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.355985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ca4b7-b8eb-43fa-ab9c-9e5dab7a37d3 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.366578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 992/1986] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:55:30.370662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ca4b7-b8eb-43fa-ab9c-9e5dab7a37d3 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.380829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ca4b7-b8eb-43fa-ab9c-9e5dab7a37d3 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.383454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b94bbc-a303-4d34-a1cf-f47696bcd928 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.385889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f366d8-47bc-49f5-b17c-5e94e1161f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.386782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f366d8-47bc-49f5-b17c-5e94e1161f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.387113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f366d8-47bc-49f5-b17c-5e94e1161f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.387418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f366d8-47bc-49f5-b17c-5e94e1161f34 admin admin] PATH_INFO: `/v3/projects/e8bb354ac12f438cbcd4d66247b55880/users/c394bd4c402c4234812b9fd7c9a8a750/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.387716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d1bcb-3463-409e-b91a-362aab60eba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.388466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1bcb-3463-409e-b91a-362aab60eba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.388757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1bcb-3463-409e-b91a-362aab60eba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.389045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1bcb-3463-409e-b91a-362aab60eba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.398864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d1bcb-3463-409e-b91a-362aab60eba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.401925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d1bcb-3463-409e-b91a-362aab60eba2 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.404217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f366d8-47bc-49f5-b17c-5e94e1161f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8bb354ac12f438cbcd4d66247b55880, user_id=c394bd4c402c4234812b9fd7c9a8a750, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.406924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f366d8-47bc-49f5-b17c-5e94e1161f34 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.411904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 995/1987] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:55:30.423712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 993/1988] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:30 2022] PUT /identity/v3/projects/e8bb354ac12f438cbcd4d66247b55880/users/c394bd4c402c4234812b9fd7c9a8a750/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:30.428344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d1bcb-3463-409e-b91a-362aab60eba2 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.439731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d1bcb-3463-409e-b91a-362aab60eba2 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.506297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119cd3d-000b-4323-bd0e-4bda8bfad3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.507053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119cd3d-000b-4323-bd0e-4bda8bfad3bb admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.507377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119cd3d-000b-4323-bd0e-4bda8bfad3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.507656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119cd3d-000b-4323-bd0e-4bda8bfad3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.508384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119cd3d-000b-4323-bd0e-4bda8bfad3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.510088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119cd3d-000b-4323-bd0e-4bda8bfad3bb admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.523697 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 996/1989] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:30.528175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f366d8-47bc-49f5-b17c-5e94e1161f34 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.538616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f366d8-47bc-49f5-b17c-5e94e1161f34 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.543582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4015117-0f31-48e7-977b-83b44750822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.544939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4015117-0f31-48e7-977b-83b44750822f admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.545366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4015117-0f31-48e7-977b-83b44750822f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.545649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4015117-0f31-48e7-977b-83b44750822f admin admin] PATH_INFO: `/v3/projects/e8bb354ac12f438cbcd4d66247b55880/users/c394bd4c402c4234812b9fd7c9a8a750/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.547706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4015117-0f31-48e7-977b-83b44750822f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8bb354ac12f438cbcd4d66247b55880, user_id=c394bd4c402c4234812b9fd7c9a8a750, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.550773 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4015117-0f31-48e7-977b-83b44750822f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.571124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 994/1990] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:30 2022] PUT /identity/v3/projects/e8bb354ac12f438cbcd4d66247b55880/users/c394bd4c402c4234812b9fd7c9a8a750/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:30.572744 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bb0c1-47d5-4464-a587-ee63ab431476 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.573085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bb0c1-47d5-4464-a587-ee63ab431476 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.573359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bb0c1-47d5-4464-a587-ee63ab431476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.608508 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-562bb0c1-47d5-4464-a587-ee63ab431476 None None] MFA Rules not processed for user `c394bd4c402c4234812b9fd7c9a8a750`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:30.610150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-562bb0c1-47d5-4464-a587-ee63ab431476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:30.685981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 997/1991] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:30.698740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4015117-0f31-48e7-977b-83b44750822f admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.701048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d4015117-0f31-48e7-977b-83b44750822f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:30.752615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:30.802412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4015117-0f31-48e7-977b-83b44750822f admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.806526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01f7f8-0e64-4eb7-90ae-af05795c0a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.807295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01f7f8-0e64-4eb7-90ae-af05795c0a30 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.807582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01f7f8-0e64-4eb7-90ae-af05795c0a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.807859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01f7f8-0e64-4eb7-90ae-af05795c0a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.818681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01f7f8-0e64-4eb7-90ae-af05795c0a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.820309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01f7f8-0e64-4eb7-90ae-af05795c0a30 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.833807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 995/1992] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:30.906678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bb0c1-47d5-4464-a587-ee63ab431476 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:30.916910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bb0c1-47d5-4464-a587-ee63ab431476 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:30.920723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa01a71-f626-4545-b97d-808b5adab39b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:30.921548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01a71-f626-4545-b97d-808b5adab39b service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:30.921824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01a71-f626-4545-b97d-808b5adab39b service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:30.922093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01a71-f626-4545-b97d-808b5adab39b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:30.956041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:30.983423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa01a71-f626-4545-b97d-808b5adab39b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:30.984898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa01a71-f626-4545-b97d-808b5adab39b service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:30.994001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 998/1993] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:34.744492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599bdbc-598e-4cca-a469-3f48bd29d285 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:34.744951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599bdbc-598e-4cca-a469-3f48bd29d285 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:34.745057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599bdbc-598e-4cca-a469-3f48bd29d285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:34.772604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-0599bdbc-598e-4cca-a469-3f48bd29d285 None None] MFA Rules not processed for user `93ae888c6e364e2e96ff52e63415a0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:34.774912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0599bdbc-598e-4cca-a469-3f48bd29d285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:34.844694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 996/1994] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:34.850350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fd619-46b5-4a7d-bd7a-28c704739644 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:34.853120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fd619-46b5-4a7d-bd7a-28c704739644 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:34.855489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fd619-46b5-4a7d-bd7a-28c704739644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:34.875758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-fb0fd619-46b5-4a7d-bd7a-28c704739644 None None] MFA Rules not processed for user `93ae888c6e364e2e96ff52e63415a0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:34.876988 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-fb0fd619-46b5-4a7d-bd7a-28c704739644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:34.884223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 999/1995] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:34.895506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599bdbc-598e-4cca-a469-3f48bd29d285 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:34.896782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0599bdbc-598e-4cca-a469-3f48bd29d285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:34.974334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599bdbc-598e-4cca-a469-3f48bd29d285 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:34.980022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede3d02-0fe0-48c7-a02c-2213b4139ae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:34.980635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede3d02-0fe0-48c7-a02c-2213b4139ae8 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:34.980945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede3d02-0fe0-48c7-a02c-2213b4139ae8 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:34.981211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede3d02-0fe0-48c7-a02c-2213b4139ae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:34.989569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede3d02-0fe0-48c7-a02c-2213b4139ae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:34.991891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede3d02-0fe0-48c7-a02c-2213b4139ae8 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:35.001818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 997/1996] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:35.709232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93510ada-517b-4d90-9c4e-9adfec072e89 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:35.709522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93510ada-517b-4d90-9c4e-9adfec072e89 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:35.710166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93510ada-517b-4d90-9c4e-9adfec072e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:35.731920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-93510ada-517b-4d90-9c4e-9adfec072e89 None None] MFA Rules not processed for user `93ae888c6e364e2e96ff52e63415a0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:35.733173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-93510ada-517b-4d90-9c4e-9adfec072e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:35.746007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1000/1997] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:41.142650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede3d02-0fe0-48c7-a02c-2213b4139ae8 service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:41.209308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:41.228430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede3d02-0fe0-48c7-a02c-2213b4139ae8 service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:41.234762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:41.235579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:41.235953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:41.236420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] PATH_INFO: `/v3/users/41a29a98d5fe48b3ab4501b86aef6061` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:41.237836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a29a98d5fe48b3ab4501b86aef6061)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:41.242319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:41.338626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a29a98d5fe48b3ab4501b86aef6061'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:41.348541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a29a98d5fe48b3ab4501b86aef6061'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:41.363322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a29a98d5fe48b3ab4501b86aef6061'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:41.364137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a29a98d5fe48b3ab4501b86aef6061'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:41.364763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a29a98d5fe48b3ab4501b86aef6061'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:41.404451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 998/1998] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:41 2022] DELETE /identity/v3/users/41a29a98d5fe48b3ab4501b86aef6061 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:41.409137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accfe84-17af-4cc7-a683-20c154bf131b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:41.409635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accfe84-17af-4cc7-a683-20c154bf131b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:41.409755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accfe84-17af-4cc7-a683-20c154bf131b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:41.435605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-8accfe84-17af-4cc7-a683-20c154bf131b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:41.437646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8accfe84-17af-4cc7-a683-20c154bf131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:41.516995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1001/1999] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:41.568220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:41.569858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:41.656394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1635604-4c3a-4724-8932-43af68a6f868 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:41.660830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c66baf-41fa-48fa-9da3-dbbca012dfcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:41.661695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c66baf-41fa-48fa-9da3-dbbca012dfcd service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:41.662133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c66baf-41fa-48fa-9da3-dbbca012dfcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:41.662808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c66baf-41fa-48fa-9da3-dbbca012dfcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:41.675960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c66baf-41fa-48fa-9da3-dbbca012dfcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:41.677584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c66baf-41fa-48fa-9da3-dbbca012dfcd service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:41.690386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 999/2000] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:41.810653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accfe84-17af-4cc7-a683-20c154bf131b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:41.812810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8accfe84-17af-4cc7-a683-20c154bf131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:41.842081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accfe84-17af-4cc7-a683-20c154bf131b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:41.848195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:41.849285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:41.849559 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:41.849831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] PATH_INFO: `/v3/users/dd9f942307f2405186d79f6e2a0acdf7` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:41.851188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd9f942307f2405186d79f6e2a0acdf7)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:41.853215 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:41.912661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd9f942307f2405186d79f6e2a0acdf7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:41.923774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd9f942307f2405186d79f6e2a0acdf7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:41.924610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd9f942307f2405186d79f6e2a0acdf7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:41.935118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd9f942307f2405186d79f6e2a0acdf7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:41.950475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd9f942307f2405186d79f6e2a0acdf7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:41.957965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1002/2001] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:41 2022] DELETE /identity/v3/users/dd9f942307f2405186d79f6e2a0acdf7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:42.162907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c66baf-41fa-48fa-9da3-dbbca012dfcd service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:42.177548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c66baf-41fa-48fa-9da3-dbbca012dfcd service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:42.184671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:42.187734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:42.230409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:42.253021 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:42.254088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:42.254370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:42.254650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] PATH_INFO: `/v3/projects/d190902b5af14ce8a1ff1710687fb6ce` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:42.255905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d190902b5af14ce8a1ff1710687fb6ce)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:42.257835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:42.270941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:42.274727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73702f-f48d-474f-94bb-1026f30b2297 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:42.280631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:42.281650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:42.281958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:42.282232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] PATH_INFO: `/v3/projects/5c7148b192884425a79290bcba1beee0` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:42.283586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c7148b192884425a79290bcba1beee0)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:42.285567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:42.320818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd190902b5af14ce8a1ff1710687fb6ce', 'request_id': 'req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.328122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd190902b5af14ce8a1ff1710687fb6ce', 'request_id': 'req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.334878 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd190902b5af14ce8a1ff1710687fb6ce', 'request_id': 'req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.341437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c7148b192884425a79290bcba1beee0', 'request_id': 'req-8488d993-7062-4aa2-91da-aaa33380ccba'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.349907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c7148b192884425a79290bcba1beee0', 'request_id': 'req-8488d993-7062-4aa2-91da-aaa33380ccba'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.353176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] The token cache is being invalidate because project d190902b5af14ce8a1ff1710687fb6ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:42.353508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.354129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.355930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1000/2002] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:42 2022] DELETE /identity/v3/projects/d190902b5af14ce8a1ff1710687fb6ce => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:42.356883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c7148b192884425a79290bcba1beee0', 'request_id': 'req-8488d993-7062-4aa2-91da-aaa33380ccba'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.361830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:42.363270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:42.377531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] The token cache is being invalidate because project 5c7148b192884425a79290bcba1beee0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:42.377873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.378525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.380474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1003/2003] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:42 2022] DELETE /identity/v3/projects/5c7148b192884425a79290bcba1beee0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:42.384349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:42.385881 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:42.434322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71fe5a-731b-4a4f-9878-51a4ffcd82fc admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:42.455684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488d993-7062-4aa2-91da-aaa33380ccba admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:42.460924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:42.462192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:42.462647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:42.462931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] PATH_INFO: `/v3/users/c394bd4c402c4234812b9fd7c9a8a750` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:42.464372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c394bd4c402c4234812b9fd7c9a8a750)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:42.466562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:42.494114 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:42.495162 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:42.496869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:42.496869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] PATH_INFO: `/v3/users/7b3a3a277aa343a5ab4f24c8ee6849af` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:42.498746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3a3a277aa343a5ab4f24c8ee6849af)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:42.501677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:42.528698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c394bd4c402c4234812b9fd7c9a8a750'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.537498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c394bd4c402c4234812b9fd7c9a8a750'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.538353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c394bd4c402c4234812b9fd7c9a8a750'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.543083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:42.548712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c394bd4c402c4234812b9fd7c9a8a750'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.563634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c394bd4c402c4234812b9fd7c9a8a750'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.564164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3a3a277aa343a5ab4f24c8ee6849af'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.566116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:42.572556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1004/2004] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:42 2022] DELETE /identity/v3/users/c394bd4c402c4234812b9fd7c9a8a750 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:42.573974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3a3a277aa343a5ab4f24c8ee6849af'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.583549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3a3a277aa343a5ab4f24c8ee6849af'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.584211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3a3a277aa343a5ab4f24c8ee6849af'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.585281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3a3a277aa343a5ab4f24c8ee6849af'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:42.606641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1001/2005] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:42 2022] DELETE /identity/v3/users/7b3a3a277aa343a5ab4f24c8ee6849af => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:42.864588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:42.866080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:42.959587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fed39-dc4c-48bc-b67c-d55e323f9e29 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.017541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.018375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.018663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.018942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] PATH_INFO: `/v3/projects/7b6fb918858246c8bd4871ea6e867480` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.020188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6fb918858246c8bd4871ea6e867480)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.022156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.053102 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.054888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:43.069368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afadcb-44ab-4c8d-a8e1-d7235fa612b6 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.075154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.076185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.076509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.076755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] PATH_INFO: `/v3/projects/e8bb354ac12f438cbcd4d66247b55880` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.078118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8bb354ac12f438cbcd4d66247b55880)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.080111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.097716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6fb918858246c8bd4871ea6e867480', 'request_id': 'req-72cf8768-bfda-4e7d-97ed-577a79c3b603'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:43.109219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6fb918858246c8bd4871ea6e867480', 'request_id': 'req-72cf8768-bfda-4e7d-97ed-577a79c3b603'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:43.129177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6fb918858246c8bd4871ea6e867480', 'request_id': 'req-72cf8768-bfda-4e7d-97ed-577a79c3b603'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:43.145514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8bb354ac12f438cbcd4d66247b55880', 'request_id': 'req-f9ef3d3a-7434-4190-b88b-8f938cba8903'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:43.146164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] The token cache is being invalidate because project 7b6fb918858246c8bd4871ea6e867480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:43.146620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:43.147431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-72cf8768-bfda-4e7d-97ed-577a79c3b603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:43.150100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1005/2006] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:42 2022] DELETE /identity/v3/projects/7b6fb918858246c8bd4871ea6e867480 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:43.155533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b35417-fd8b-4eb8-a7d6-098f6c5cbad1 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.155870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b35417-fd8b-4eb8-a7d6-098f6c5cbad1 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.156173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b35417-fd8b-4eb8-a7d6-098f6c5cbad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.157034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8bb354ac12f438cbcd4d66247b55880', 'request_id': 'req-f9ef3d3a-7434-4190-b88b-8f938cba8903'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:43.164973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8bb354ac12f438cbcd4d66247b55880', 'request_id': 'req-f9ef3d3a-7434-4190-b88b-8f938cba8903'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:43.176092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d3b35417-fd8b-4eb8-a7d6-098f6c5cbad1 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:43.177300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d3b35417-fd8b-4eb8-a7d6-098f6c5cbad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:43.181007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] The token cache is being invalidate because project e8bb354ac12f438cbcd4d66247b55880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:43.181355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:43.181995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f9ef3d3a-7434-4190-b88b-8f938cba8903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:43.183798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1002/2007] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:43 2022] DELETE /identity/v3/projects/e8bb354ac12f438cbcd4d66247b55880 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:43.187710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:43.188722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea10f6-dc88-4e46-bab5-4c35e6a4cac7 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.188977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea10f6-dc88-4e46-bab5-4c35e6a4cac7 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.189252 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea10f6-dc88-4e46-bab5-4c35e6a4cac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.208253 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d2ea10f6-dc88-4e46-bab5-4c35e6a4cac7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:43.209547 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d2ea10f6-dc88-4e46-bab5-4c35e6a4cac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:43.238947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1006/2008] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:43.246937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b5f03-c7a7-4bfb-b38d-e9e522480a6e None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.247226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b5f03-c7a7-4bfb-b38d-e9e522480a6e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.247534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b5f03-c7a7-4bfb-b38d-e9e522480a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.266549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-9f9b5f03-c7a7-4bfb-b38d-e9e522480a6e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:43.267869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9f9b5f03-c7a7-4bfb-b38d-e9e522480a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:43.269896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1003/2009] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:43.275860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1007/2010] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:43.278312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7d48-827d-4dc4-bbaf-07e67d87aae1 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.278457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7d48-827d-4dc4-bbaf-07e67d87aae1 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.278681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7d48-827d-4dc4-bbaf-07e67d87aae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.279735 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b5f03-c7a7-4bfb-b38d-e9e522480a6e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.289372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b5f03-c7a7-4bfb-b38d-e9e522480a6e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.294387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9db67-de97-4a1b-8858-ee40403f4b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.295036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9db67-de97-4a1b-8858-ee40403f4b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.295426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9db67-de97-4a1b-8858-ee40403f4b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.295699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9db67-de97-4a1b-8858-ee40403f4b2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.296391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-8d7f7d48-827d-4dc4-bbaf-07e67d87aae1 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:43.296499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9db67-de97-4a1b-8858-ee40403f4b2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:55:43.296733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9db67-de97-4a1b-8858-ee40403f4b2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.297585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8d7f7d48-827d-4dc4-bbaf-07e67d87aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:43.298737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9db67-de97-4a1b-8858-ee40403f4b2a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.306306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1004/2011] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:43.310788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7d48-827d-4dc4-bbaf-07e67d87aae1 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.315929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1008/2012] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:55:43.317107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9db67-de97-4a1b-8858-ee40403f4b2a admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.320362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7d48-827d-4dc4-bbaf-07e67d87aae1 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.325847 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e452bf7-d217-4af6-aa04-dfcce217f6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.326519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e452bf7-d217-4af6-aa04-dfcce217f6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.326669 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9db67-de97-4a1b-8858-ee40403f4b2a admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.326805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e452bf7-d217-4af6-aa04-dfcce217f6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.327141 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e452bf7-d217-4af6-aa04-dfcce217f6b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.327977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e452bf7-d217-4af6-aa04-dfcce217f6b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:55:43.328286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e452bf7-d217-4af6-aa04-dfcce217f6b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.329905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e452bf7-d217-4af6-aa04-dfcce217f6b9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.331292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cfe88-8928-4d74-9fd5-b7248dd002ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.331932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cfe88-8928-4d74-9fd5-b7248dd002ca admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.332205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cfe88-8928-4d74-9fd5-b7248dd002ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.332477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cfe88-8928-4d74-9fd5-b7248dd002ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.333158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cfe88-8928-4d74-9fd5-b7248dd002ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.335061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cfe88-8928-4d74-9fd5-b7248dd002ca admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.346315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1005/2013] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:55:43.347394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e452bf7-d217-4af6-aa04-dfcce217f6b9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.357334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e452bf7-d217-4af6-aa04-dfcce217f6b9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.366077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1009/2014] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:43.366931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cfe88-8928-4d74-9fd5-b7248dd002ca admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.372154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6280201-2a70-48d5-ae64-24435cffb6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.372929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6280201-2a70-48d5-ae64-24435cffb6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.373222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6280201-2a70-48d5-ae64-24435cffb6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.373499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6280201-2a70-48d5-ae64-24435cffb6d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.374466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6280201-2a70-48d5-ae64-24435cffb6d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.376778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6280201-2a70-48d5-ae64-24435cffb6d7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.406828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1006/2015] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:55:43.411226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6280201-2a70-48d5-ae64-24435cffb6d7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.421783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:43.446391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cfe88-8928-4d74-9fd5-b7248dd002ca admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.483007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6280201-2a70-48d5-ae64-24435cffb6d7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.490275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820237d-4d27-40f7-bd80-343d009d8b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.490982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820237d-4d27-40f7-bd80-343d009d8b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.491338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820237d-4d27-40f7-bd80-343d009d8b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.491569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820237d-4d27-40f7-bd80-343d009d8b2c admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.492284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820237d-4d27-40f7-bd80-343d009d8b2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.494231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820237d-4d27-40f7-bd80-343d009d8b2c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.515646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4afe5e-67bf-4799-ad77-bf3272cbf705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.516339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4afe5e-67bf-4799-ad77-bf3272cbf705 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.516615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4afe5e-67bf-4799-ad77-bf3272cbf705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.516887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4afe5e-67bf-4799-ad77-bf3272cbf705 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.517940 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4afe5e-67bf-4799-ad77-bf3272cbf705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.520013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4afe5e-67bf-4799-ad77-bf3272cbf705 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.521739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1007/2016] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:43.525768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820237d-4d27-40f7-bd80-343d009d8b2c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.535799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820237d-4d27-40f7-bd80-343d009d8b2c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.541394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05240709-8a66-438b-8e2b-ea3b48bde719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.542218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240709-8a66-438b-8e2b-ea3b48bde719 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.542505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240709-8a66-438b-8e2b-ea3b48bde719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.542855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240709-8a66-438b-8e2b-ea3b48bde719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.543639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05240709-8a66-438b-8e2b-ea3b48bde719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.546252 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05240709-8a66-438b-8e2b-ea3b48bde719 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.547631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1010/2017] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:43.552310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4afe5e-67bf-4799-ad77-bf3272cbf705 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.565156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4afe5e-67bf-4799-ad77-bf3272cbf705 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.570174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1008/2018] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:55:43.570276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f0c64-69b8-4ac8-84c3-268a39800ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.570872 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f0c64-69b8-4ac8-84c3-268a39800ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.571133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f0c64-69b8-4ac8-84c3-268a39800ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.571451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f0c64-69b8-4ac8-84c3-268a39800ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.572184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f0c64-69b8-4ac8-84c3-268a39800ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.573970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05240709-8a66-438b-8e2b-ea3b48bde719 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.574206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f0c64-69b8-4ac8-84c3-268a39800ad9 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.581750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:43.586292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05240709-8a66-438b-8e2b-ea3b48bde719 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.590496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1011/2019] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:55:43.591523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f0c64-69b8-4ac8-84c3-268a39800ad9 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.592719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb295c7-6f35-4b05-83b7-4af20d10f72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.593428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb295c7-6f35-4b05-83b7-4af20d10f72b admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.593711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb295c7-6f35-4b05-83b7-4af20d10f72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.593987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb295c7-6f35-4b05-83b7-4af20d10f72b admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.594715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb295c7-6f35-4b05-83b7-4af20d10f72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.596552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb295c7-6f35-4b05-83b7-4af20d10f72b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.605086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f0c64-69b8-4ac8-84c3-268a39800ad9 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.611055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1009/2020] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:55:43.612248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb295c7-6f35-4b05-83b7-4af20d10f72b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.613550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c230ed9-b825-4fe6-b80f-970741dc8b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.614332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230ed9-b825-4fe6-b80f-970741dc8b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.614747 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230ed9-b825-4fe6-b80f-970741dc8b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.615082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230ed9-b825-4fe6-b80f-970741dc8b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.615913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c230ed9-b825-4fe6-b80f-970741dc8b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.617641 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c230ed9-b825-4fe6-b80f-970741dc8b44 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.623168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb295c7-6f35-4b05-83b7-4af20d10f72b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.628308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68eaa8-2377-4949-9713-f194e36af1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.629398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68eaa8-2377-4949-9713-f194e36af1af admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.629749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68eaa8-2377-4949-9713-f194e36af1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.630088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68eaa8-2377-4949-9713-f194e36af1af admin admin] PATH_INFO: `/v3/projects/06e99da8a96940b196ae230dc3fd67ee/users/ef086a7968fe4733af361e1d8436ef89/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.630774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1012/2021] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:43.634726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c230ed9-b825-4fe6-b80f-970741dc8b44 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.645891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c230ed9-b825-4fe6-b80f-970741dc8b44 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.651527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fab8dc-ada8-4b45-b646-bc2bd7ab1475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.652558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fab8dc-ada8-4b45-b646-bc2bd7ab1475 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.652920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68eaa8-2377-4949-9713-f194e36af1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e99da8a96940b196ae230dc3fd67ee, user_id=ef086a7968fe4733af361e1d8436ef89, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.653023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fab8dc-ada8-4b45-b646-bc2bd7ab1475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.653352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fab8dc-ada8-4b45-b646-bc2bd7ab1475 admin admin] PATH_INFO: `/v3/projects/471f30df1e0c43aab849b9c010009142/users/b02288fe1cbd4328b0e567b9c6832059/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.655987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68eaa8-2377-4949-9713-f194e36af1af admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.670160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fab8dc-ada8-4b45-b646-bc2bd7ab1475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=471f30df1e0c43aab849b9c010009142, user_id=b02288fe1cbd4328b0e567b9c6832059, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.673210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fab8dc-ada8-4b45-b646-bc2bd7ab1475 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.673351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1010/2022] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:43 2022] PUT /identity/v3/projects/06e99da8a96940b196ae230dc3fd67ee/users/ef086a7968fe4733af361e1d8436ef89/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:43.685080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c371e3-a5bf-4184-a60c-75ae17428ec3 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.685373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c371e3-a5bf-4184-a60c-75ae17428ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.685704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c371e3-a5bf-4184-a60c-75ae17428ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.695544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1013/2023] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:43 2022] PUT /identity/v3/projects/471f30df1e0c43aab849b9c010009142/users/b02288fe1cbd4328b0e567b9c6832059/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:43.706213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375de7d0-29e2-4275-a559-8b9663559ead None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.706514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375de7d0-29e2-4275-a559-8b9663559ead None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.706782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375de7d0-29e2-4275-a559-8b9663559ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.717893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-63c371e3-a5bf-4184-a60c-75ae17428ec3 None None] MFA Rules not processed for user `ef086a7968fe4733af361e1d8436ef89`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:43.719274 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-63c371e3-a5bf-4184-a60c-75ae17428ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:43.732736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-375de7d0-29e2-4275-a559-8b9663559ead None None] MFA Rules not processed for user `b02288fe1cbd4328b0e567b9c6832059`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:43.734148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-375de7d0-29e2-4275-a559-8b9663559ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:43.800967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1011/2024] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:43.806918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c371e3-a5bf-4184-a60c-75ae17428ec3 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.810665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1014/2025] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:43.823585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375de7d0-29e2-4275-a559-8b9663559ead None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:43.825259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-375de7d0-29e2-4275-a559-8b9663559ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:43.862241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:43.886556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c371e3-a5bf-4184-a60c-75ae17428ec3 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.891846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c1c01-f5bb-4862-b87c-487baa6295e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.892527 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1c01-f5bb-4862-b87c-487baa6295e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.892811 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1c01-f5bb-4862-b87c-487baa6295e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.893086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1c01-f5bb-4862-b87c-487baa6295e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.894447 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c1c01-f5bb-4862-b87c-487baa6295e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.895944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c1c01-f5bb-4862-b87c-487baa6295e9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.899239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375de7d0-29e2-4275-a559-8b9663559ead None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:43.903313 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a64fa7-153b-4ab5-bef9-782830dc5bba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab33473d40094ec681e538cde250ec2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:43.904015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64fa7-153b-4ab5-bef9-782830dc5bba service glance] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:43.905074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64fa7-153b-4ab5-bef9-782830dc5bba service glance] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:43.905702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64fa7-153b-4ab5-bef9-782830dc5bba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:43.916508 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a64fa7-153b-4ab5-bef9-782830dc5bba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:43.918412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a64fa7-153b-4ab5-bef9-782830dc5bba service glance] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:43.929483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1012/2026] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:43.930338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1015/2027] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:55:43.934803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c1c01-f5bb-4862-b87c-487baa6295e9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:44.013332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c1c01-f5bb-4862-b87c-487baa6295e9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:44.018231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40980b2-b521-427d-b29c-f46e342588e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:44.018953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40980b2-b521-427d-b29c-f46e342588e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:44.019230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40980b2-b521-427d-b29c-f46e342588e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:44.019533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40980b2-b521-427d-b29c-f46e342588e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:44.020351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40980b2-b521-427d-b29c-f46e342588e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:44.022787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40980b2-b521-427d-b29c-f46e342588e3 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:44.050114 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1013/2028] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:43 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:44.054057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a64fa7-153b-4ab5-bef9-782830dc5bba service glance] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:44.065281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a64fa7-153b-4ab5-bef9-782830dc5bba service glance] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:44.070431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3a9c6-85eb-42d9-ab89-f2736d2aa1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:44.071177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3a9c6-85eb-42d9-ab89-f2736d2aa1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:44.071471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3a9c6-85eb-42d9-ab89-f2736d2aa1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:44.071749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3a9c6-85eb-42d9-ab89-f2736d2aa1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:44.072598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3a9c6-85eb-42d9-ab89-f2736d2aa1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:44.074878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3a9c6-85eb-42d9-ab89-f2736d2aa1f5 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:44.089908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1016/2029] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:44.090617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40980b2-b521-427d-b29c-f46e342588e3 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:44.101885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40980b2-b521-427d-b29c-f46e342588e3 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:44.107869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b65f56-63ae-46f3-8558-f834c15ea0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:44.108801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b65f56-63ae-46f3-8558-f834c15ea0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:44.109086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b65f56-63ae-46f3-8558-f834c15ea0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:44.109366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b65f56-63ae-46f3-8558-f834c15ea0cc admin admin] PATH_INFO: `/v3/projects/46717ded483e48ba8da3d4e9817f47d1/users/3e445b9b74494f47878b0d3878ae7f39/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:44.126145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b65f56-63ae-46f3-8558-f834c15ea0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=46717ded483e48ba8da3d4e9817f47d1, user_id=3e445b9b74494f47878b0d3878ae7f39, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:44.128950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b65f56-63ae-46f3-8558-f834c15ea0cc admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:44.146113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1014/2030] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:44 2022] PUT /identity/v3/projects/46717ded483e48ba8da3d4e9817f47d1/users/3e445b9b74494f47878b0d3878ae7f39/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:44.149969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3a9c6-85eb-42d9-ab89-f2736d2aa1f5 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:44.164225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3a9c6-85eb-42d9-ab89-f2736d2aa1f5 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:44.220925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:44.237902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd4fb5-f0d7-46af-a75f-dde7097e3f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:44.238600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd4fb5-f0d7-46af-a75f-dde7097e3f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:44.238884 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd4fb5-f0d7-46af-a75f-dde7097e3f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:44.239213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd4fb5-f0d7-46af-a75f-dde7097e3f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:44.239992 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd4fb5-f0d7-46af-a75f-dde7097e3f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:44.241653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd4fb5-f0d7-46af-a75f-dde7097e3f57 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:44.256452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1017/2031] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:44.258113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b65f56-63ae-46f3-8558-f834c15ea0cc admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:44.269671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b65f56-63ae-46f3-8558-f834c15ea0cc admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:44.275012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09785a1b-c4bd-42f3-b472-f98a3aa58d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:44.276541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785a1b-c4bd-42f3-b472-f98a3aa58d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:44.276821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785a1b-c4bd-42f3-b472-f98a3aa58d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:44.277099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785a1b-c4bd-42f3-b472-f98a3aa58d42 admin admin] PATH_INFO: `/v3/projects/46717ded483e48ba8da3d4e9817f47d1/users/3e445b9b74494f47878b0d3878ae7f39/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:44.280053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09785a1b-c4bd-42f3-b472-f98a3aa58d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46717ded483e48ba8da3d4e9817f47d1, user_id=3e445b9b74494f47878b0d3878ae7f39, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:44.283228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09785a1b-c4bd-42f3-b472-f98a3aa58d42 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:44.297439 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1015/2032] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:44 2022] PUT /identity/v3/projects/46717ded483e48ba8da3d4e9817f47d1/users/3e445b9b74494f47878b0d3878ae7f39/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:44.310870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadae32-462a-4efd-878e-814bccdff28b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:44.312482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadae32-462a-4efd-878e-814bccdff28b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:44.313574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadae32-462a-4efd-878e-814bccdff28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:44.347073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-3cadae32-462a-4efd-878e-814bccdff28b None None] MFA Rules not processed for user `3e445b9b74494f47878b0d3878ae7f39`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:44.348628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3cadae32-462a-4efd-878e-814bccdff28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:44.437959 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1018/2033] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:44 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:44.456937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09785a1b-c4bd-42f3-b472-f98a3aa58d42 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:44.459912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-09785a1b-c4bd-42f3-b472-f98a3aa58d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:44.542523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09785a1b-c4bd-42f3-b472-f98a3aa58d42 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:44.546548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d01bf7-87ee-4ac9-803d-a98ae4ca3721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:44.547230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d01bf7-87ee-4ac9-803d-a98ae4ca3721 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:44.547542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d01bf7-87ee-4ac9-803d-a98ae4ca3721 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:44.548165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d01bf7-87ee-4ac9-803d-a98ae4ca3721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:44.558196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d01bf7-87ee-4ac9-803d-a98ae4ca3721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:44.559874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d01bf7-87ee-4ac9-803d-a98ae4ca3721 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:44.570106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1016/2034] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:44 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:44.838014 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadae32-462a-4efd-878e-814bccdff28b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:44.839510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3cadae32-462a-4efd-878e-814bccdff28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:44.943849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadae32-462a-4efd-878e-814bccdff28b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:44.949651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:44.950790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:44.951142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:44.951512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] PATH_INFO: `/v3/users/6a391589707442d9b8a0d14ada7f1b08` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:44.952883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a391589707442d9b8a0d14ada7f1b08)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:44.954897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:45.055077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a391589707442d9b8a0d14ada7f1b08'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:45.068389 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a391589707442d9b8a0d14ada7f1b08'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:45.069811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a391589707442d9b8a0d14ada7f1b08'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:45.102263 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a391589707442d9b8a0d14ada7f1b08'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:45.117223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:45.128631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a391589707442d9b8a0d14ada7f1b08'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:45.136237 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1019/2035] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:44 2022] DELETE /identity/v3/users/6a391589707442d9b8a0d14ada7f1b08 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:45.141034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94e898-4a60-4fc0-9b85-8a8b65537da1 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:45.141418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94e898-4a60-4fc0-9b85-8a8b65537da1 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:45.141793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94e898-4a60-4fc0-9b85-8a8b65537da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:45.171372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ae94e898-4a60-4fc0-9b85-8a8b65537da1 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:45.173317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ae94e898-4a60-4fc0-9b85-8a8b65537da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:45.198062 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:45.280230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1017/2036] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:45.288806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:45.294545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:45.382001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac70cd2-be61-4a7f-a33b-2c8080fa5126 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:45.386457 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2463cd-2472-4157-88e3-0a681d8bbb2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:45.387351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2463cd-2472-4157-88e3-0a681d8bbb2b service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:45.387736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2463cd-2472-4157-88e3-0a681d8bbb2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:45.388163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2463cd-2472-4157-88e3-0a681d8bbb2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:45.403295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2463cd-2472-4157-88e3-0a681d8bbb2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:45.405048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2463cd-2472-4157-88e3-0a681d8bbb2b service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:45.415675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1020/2037] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:45.730014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94e898-4a60-4fc0-9b85-8a8b65537da1 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:45.732649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ae94e898-4a60-4fc0-9b85-8a8b65537da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:45.755630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94e898-4a60-4fc0-9b85-8a8b65537da1 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:45.760394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:45.761284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:45.761604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:45.762329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] PATH_INFO: `/v3/projects/2f0809822766482da031a81e83c6e53a` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:45.763224 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0809822766482da031a81e83c6e53a)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:45.765677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:45.837502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0809822766482da031a81e83c6e53a', 'request_id': 'req-2451e089-fffe-4d1e-ae03-79c83b7b9905'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:45.847951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0809822766482da031a81e83c6e53a', 'request_id': 'req-2451e089-fffe-4d1e-ae03-79c83b7b9905'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:45.858429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0809822766482da031a81e83c6e53a', 'request_id': 'req-2451e089-fffe-4d1e-ae03-79c83b7b9905'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:45.872668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] The token cache is being invalidate because project 2f0809822766482da031a81e83c6e53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:45.873181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:45.873919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-2451e089-fffe-4d1e-ae03-79c83b7b9905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:45.875529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1018/2038] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:45 2022] DELETE /identity/v3/projects/2f0809822766482da031a81e83c6e53a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:45.880472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214018f-1dc0-43a5-a9dd-61179145a957 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:45.880927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214018f-1dc0-43a5-a9dd-61179145a957 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:45.881085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214018f-1dc0-43a5-a9dd-61179145a957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:45.898564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-5214018f-1dc0-43a5-a9dd-61179145a957 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:45.899849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5214018f-1dc0-43a5-a9dd-61179145a957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:45.965378 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1021/2039] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:45.974112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1758f-cbdf-4e0b-9004-f367c60f72f9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:45.974398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1758f-cbdf-4e0b-9004-f367c60f72f9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:45.974668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1758f-cbdf-4e0b-9004-f367c60f72f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:45.996860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-97b1758f-cbdf-4e0b-9004-f367c60f72f9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:45.998081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-97b1758f-cbdf-4e0b-9004-f367c60f72f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:46.008952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1019/2040] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:46.013305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214018f-1dc0-43a5-a9dd-61179145a957 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.023019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214018f-1dc0-43a5-a9dd-61179145a957 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.027796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb88e26-f131-4439-a302-3dc6fa54867e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.028463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb88e26-f131-4439-a302-3dc6fa54867e admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.028746 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb88e26-f131-4439-a302-3dc6fa54867e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.029024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb88e26-f131-4439-a302-3dc6fa54867e admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.029818 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb88e26-f131-4439-a302-3dc6fa54867e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:55:46.030126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb88e26-f131-4439-a302-3dc6fa54867e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.031771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb88e26-f131-4439-a302-3dc6fa54867e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.044332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1022/2041] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:55:46.049396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1758f-cbdf-4e0b-9004-f367c60f72f9 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.059434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1758f-cbdf-4e0b-9004-f367c60f72f9 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.064051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b6265-91fe-4192-83a5-d34d1022617e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.064649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b6265-91fe-4192-83a5-d34d1022617e admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.064935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b6265-91fe-4192-83a5-d34d1022617e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.065204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b6265-91fe-4192-83a5-d34d1022617e admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.065975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b6265-91fe-4192-83a5-d34d1022617e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.067964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b6265-91fe-4192-83a5-d34d1022617e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.097760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1020/2042] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:46 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:46.102882 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb88e26-f131-4439-a302-3dc6fa54867e admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.115997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb88e26-f131-4439-a302-3dc6fa54867e admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.181350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9af200-a81d-447b-8ba7-df7e8892c848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.182043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9af200-a81d-447b-8ba7-df7e8892c848 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.182440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9af200-a81d-447b-8ba7-df7e8892c848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.182719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9af200-a81d-447b-8ba7-df7e8892c848 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.183668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9af200-a81d-447b-8ba7-df7e8892c848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.185597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9af200-a81d-447b-8ba7-df7e8892c848 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.210488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1023/2043] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:46 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:46.213635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b6265-91fe-4192-83a5-d34d1022617e admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.224766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b6265-91fe-4192-83a5-d34d1022617e admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.231120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9af200-a81d-447b-8ba7-df7e8892c848 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.232556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bf0eb-2c7d-47e9-a733-acf2e586b21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.232866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-dd9af200-a81d-447b-8ba7-df7e8892c848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:46.233239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bf0eb-2c7d-47e9-a733-acf2e586b21b admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.233523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bf0eb-2c7d-47e9-a733-acf2e586b21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.233824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bf0eb-2c7d-47e9-a733-acf2e586b21b admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.234537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bf0eb-2c7d-47e9-a733-acf2e586b21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.236180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bf0eb-2c7d-47e9-a733-acf2e586b21b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.244586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9af200-a81d-447b-8ba7-df7e8892c848 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.248335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1021/2044] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:46.249238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.250134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.250415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.250686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] PATH_INFO: `/v3/users/ef086a7968fe4733af361e1d8436ef89` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.252030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bf0eb-2c7d-47e9-a733-acf2e586b21b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.252186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef086a7968fe4733af361e1d8436ef89)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.254118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.255768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:46.264888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bf0eb-2c7d-47e9-a733-acf2e586b21b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.270190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd70f58-3b3c-45b4-8281-f78c2aea8fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.270888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70f58-3b3c-45b4-8281-f78c2aea8fec admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.271171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70f58-3b3c-45b4-8281-f78c2aea8fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.271480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70f58-3b3c-45b4-8281-f78c2aea8fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.272208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd70f58-3b3c-45b4-8281-f78c2aea8fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.274507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd70f58-3b3c-45b4-8281-f78c2aea8fec admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.284885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1022/2045] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:55:46.289950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd70f58-3b3c-45b4-8281-f78c2aea8fec admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.306472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd70f58-3b3c-45b4-8281-f78c2aea8fec admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.313511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e10fd1-0bf2-4f08-96ac-753f9f02793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.314915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e10fd1-0bf2-4f08-96ac-753f9f02793f admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.315183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e10fd1-0bf2-4f08-96ac-753f9f02793f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.315463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e10fd1-0bf2-4f08-96ac-753f9f02793f admin admin] PATH_INFO: `/v3/projects/bde733786e2a48278fc6c4193c0d2adb/users/5c9e99b960d240e787e4284450ad887f/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.325271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef086a7968fe4733af361e1d8436ef89'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:46.331218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e10fd1-0bf2-4f08-96ac-753f9f02793f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde733786e2a48278fc6c4193c0d2adb, user_id=5c9e99b960d240e787e4284450ad887f, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.333567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef086a7968fe4733af361e1d8436ef89'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:46.334109 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e10fd1-0bf2-4f08-96ac-753f9f02793f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.334253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef086a7968fe4733af361e1d8436ef89'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:46.348185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef086a7968fe4733af361e1d8436ef89'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:46.348796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1023/2046] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:46 2022] PUT /identity/v3/projects/bde733786e2a48278fc6c4193c0d2adb/users/5c9e99b960d240e787e4284450ad887f/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:46.360179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041697db-a821-4d75-8bb3-f53b3c0d7b13 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.360467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041697db-a821-4d75-8bb3-f53b3c0d7b13 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.360745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041697db-a821-4d75-8bb3-f53b3c0d7b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.367065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-52c84457-3622-4160-90d5-ed0442e2773e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef086a7968fe4733af361e1d8436ef89'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:46.375700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1024/2047] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:46 2022] DELETE /identity/v3/users/ef086a7968fe4733af361e1d8436ef89 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:46.381093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b65c18-491f-498c-8120-5302740e93d9 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.381093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b65c18-491f-498c-8120-5302740e93d9 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.381093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b65c18-491f-498c-8120-5302740e93d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.390167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-041697db-a821-4d75-8bb3-f53b3c0d7b13 None None] MFA Rules not processed for user `5c9e99b960d240e787e4284450ad887f`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:46.391542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-041697db-a821-4d75-8bb3-f53b3c0d7b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:46.399899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-e7b65c18-491f-498c-8120-5302740e93d9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:46.401109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e7b65c18-491f-498c-8120-5302740e93d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:46.413835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:46.480841 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1024/2048] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:46.485571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041697db-a821-4d75-8bb3-f53b3c0d7b13 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.486900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-041697db-a821-4d75-8bb3-f53b3c0d7b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:46.490774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1025/2049] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:46.497810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b65c18-491f-498c-8120-5302740e93d9 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.499159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e7b65c18-491f-498c-8120-5302740e93d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:46.504829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041697db-a821-4d75-8bb3-f53b3c0d7b13 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.510168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2da58-a08c-4b80-b182-d3bcf32d0a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.510843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2da58-a08c-4b80-b182-d3bcf32d0a00 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.511123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2da58-a08c-4b80-b182-d3bcf32d0a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.511614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2da58-a08c-4b80-b182-d3bcf32d0a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.512593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2da58-a08c-4b80-b182-d3bcf32d0a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.514518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2da58-a08c-4b80-b182-d3bcf32d0a00 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.544503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1025/2050] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:46 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:55:46.549052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2da58-a08c-4b80-b182-d3bcf32d0a00 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.557259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b65c18-491f-498c-8120-5302740e93d9 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.598290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:46.623102 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2da58-a08c-4b80-b182-d3bcf32d0a00 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.623821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f7a6e-850d-439b-bca0-3bdba86ccbe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.624511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f7a6e-850d-439b-bca0-3bdba86ccbe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.624788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f7a6e-850d-439b-bca0-3bdba86ccbe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.625283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f7a6e-850d-439b-bca0-3bdba86ccbe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.628228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1e7c3-7eae-4461-a336-5af9ca27a159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.628912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1e7c3-7eae-4461-a336-5af9ca27a159 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.629194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1e7c3-7eae-4461-a336-5af9ca27a159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.629469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1e7c3-7eae-4461-a336-5af9ca27a159 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.630289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1e7c3-7eae-4461-a336-5af9ca27a159 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.632375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1e7c3-7eae-4461-a336-5af9ca27a159 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.642444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f7a6e-850d-439b-bca0-3bdba86ccbe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.643517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f7a6e-850d-439b-bca0-3bdba86ccbe5 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.654819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1026/2051] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:55:46.655855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1026/2052] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:46 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:55:46.659608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1e7c3-7eae-4461-a336-5af9ca27a159 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.669616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1e7c3-7eae-4461-a336-5af9ca27a159 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.674814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9b05d-43ff-4bca-b479-24a43e7b23c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.675551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9b05d-43ff-4bca-b479-24a43e7b23c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.675872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9b05d-43ff-4bca-b479-24a43e7b23c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.676134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9b05d-43ff-4bca-b479-24a43e7b23c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.676864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9b05d-43ff-4bca-b479-24a43e7b23c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.678531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9b05d-43ff-4bca-b479-24a43e7b23c8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.693793 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1027/2053] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:55:46.698076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f7a6e-850d-439b-bca0-3bdba86ccbe5 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.708857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f7a6e-850d-439b-bca0-3bdba86ccbe5 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.715199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfe859-b97f-4fe2-9fc6-3d3969255afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.716283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfe859-b97f-4fe2-9fc6-3d3969255afe admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.716740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfe859-b97f-4fe2-9fc6-3d3969255afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.717010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfe859-b97f-4fe2-9fc6-3d3969255afe admin admin] PATH_INFO: `/v3/projects/f317a0144b7e4c4a972281f20dc8505f/users/2c142fad085f415290289dd5ef267d97/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.733431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfe859-b97f-4fe2-9fc6-3d3969255afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f317a0144b7e4c4a972281f20dc8505f, user_id=2c142fad085f415290289dd5ef267d97, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.736203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfe859-b97f-4fe2-9fc6-3d3969255afe admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.750940 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1027/2054] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:46 2022] PUT /identity/v3/projects/f317a0144b7e4c4a972281f20dc8505f/users/2c142fad085f415290289dd5ef267d97/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:46.754768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9b05d-43ff-4bca-b479-24a43e7b23c8 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.766522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9b05d-43ff-4bca-b479-24a43e7b23c8 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.830239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7277a-6291-428c-8723-b3ab356d3168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.830974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7277a-6291-428c-8723-b3ab356d3168 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.831267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7277a-6291-428c-8723-b3ab356d3168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.831560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7277a-6291-428c-8723-b3ab356d3168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.832598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7277a-6291-428c-8723-b3ab356d3168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.833985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7277a-6291-428c-8723-b3ab356d3168 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.844372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1028/2055] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:46.848589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfe859-b97f-4fe2-9fc6-3d3969255afe admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.859082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfe859-b97f-4fe2-9fc6-3d3969255afe admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:46.863809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbe662-3794-4ad4-ac05-11386f4337ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:46.864698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbe662-3794-4ad4-ac05-11386f4337ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.864974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbe662-3794-4ad4-ac05-11386f4337ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.865244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbe662-3794-4ad4-ac05-11386f4337ca admin admin] PATH_INFO: `/v3/projects/f317a0144b7e4c4a972281f20dc8505f/users/2c142fad085f415290289dd5ef267d97/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.867260 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbe662-3794-4ad4-ac05-11386f4337ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f317a0144b7e4c4a972281f20dc8505f, user_id=2c142fad085f415290289dd5ef267d97, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:46.869904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbe662-3794-4ad4-ac05-11386f4337ca admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:46.880771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1028/2056] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:46 2022] PUT /identity/v3/projects/f317a0144b7e4c4a972281f20dc8505f/users/2c142fad085f415290289dd5ef267d97/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:46.891922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6ca2-1f2e-41b5-8fed-9486469e1713 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:46.892159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6ca2-1f2e-41b5-8fed-9486469e1713 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:46.892504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6ca2-1f2e-41b5-8fed-9486469e1713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:46.925678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:46.930172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-6d9b6ca2-1f2e-41b5-8fed-9486469e1713 None None] MFA Rules not processed for user `2c142fad085f415290289dd5ef267d97`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:46.932841 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6d9b6ca2-1f2e-41b5-8fed-9486469e1713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:46.949642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbe662-3794-4ad4-ac05-11386f4337ca admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:46.951497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e4fbe662-3794-4ad4-ac05-11386f4337ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:47.010356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1029/2057] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:47.025414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b6ca2-1f2e-41b5-8fed-9486469e1713 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:47.026770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6d9b6ca2-1f2e-41b5-8fed-9486469e1713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:47.035623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbe662-3794-4ad4-ac05-11386f4337ca admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:47.040481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:47.041486 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:47.041827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:47.042327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] PATH_INFO: `/v3/projects/06e99da8a96940b196ae230dc3fd67ee` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:47.044046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e99da8a96940b196ae230dc3fd67ee)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:47.047211 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:47.109857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b6ca2-1f2e-41b5-8fed-9486469e1713 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:47.117083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e99da8a96940b196ae230dc3fd67ee', 'request_id': 'req-29109f83-a522-402e-b008-21ef4ca0434f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:47.124489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e99da8a96940b196ae230dc3fd67ee', 'request_id': 'req-29109f83-a522-402e-b008-21ef4ca0434f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:47.131455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e99da8a96940b196ae230dc3fd67ee', 'request_id': 'req-29109f83-a522-402e-b008-21ef4ca0434f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:47.149302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] The token cache is being invalidate because project 06e99da8a96940b196ae230dc3fd67ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:47.149830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:47.150573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:47.153537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1029/2058] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:46 2022] DELETE /identity/v3/projects/06e99da8a96940b196ae230dc3fd67ee => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:47.157136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:47.158538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:47.166869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487eec-9478-4615-b589-3a50e7759746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:47.167605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487eec-9478-4615-b589-3a50e7759746 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:47.168344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487eec-9478-4615-b589-3a50e7759746 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:47.168344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487eec-9478-4615-b589-3a50e7759746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:47.171132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-02487eec-9478-4615-b589-3a50e7759746 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:47.211727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:47.250505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29109f83-a522-402e-b008-21ef4ca0434f admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:47.255411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:47.256404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:47.256752 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:47.257109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] PATH_INFO: `/v3/users/3e445b9b74494f47878b0d3878ae7f39` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:47.258406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e445b9b74494f47878b0d3878ae7f39)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:47.260433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:47.260675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02487eec-9478-4615-b589-3a50e7759746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:47.262228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02487eec-9478-4615-b589-3a50e7759746 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:47.279518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1030/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:55:47.316832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e445b9b74494f47878b0d3878ae7f39'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:47.336111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e445b9b74494f47878b0d3878ae7f39'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:47.337034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e445b9b74494f47878b0d3878ae7f39'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:47.363296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e445b9b74494f47878b0d3878ae7f39'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:47.378259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-491c72ce-cd7a-43fb-889e-7e2008483bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e445b9b74494f47878b0d3878ae7f39'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:47.387784 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1030/2060] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:47 2022] DELETE /identity/v3/users/3e445b9b74494f47878b0d3878ae7f39 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:55:47.860885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487eec-9478-4615-b589-3a50e7759746 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:47.873560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487eec-9478-4615-b589-3a50e7759746 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:47.879175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:47.933552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:47.934460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:47.934741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:47.935004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] PATH_INFO: `/v3/projects/46717ded483e48ba8da3d4e9817f47d1` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:47.936243 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] RBAC: Authorizing `identity:delete_project(project_id=46717ded483e48ba8da3d4e9817f47d1)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:47.938144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:47.994113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46717ded483e48ba8da3d4e9817f47d1', 'request_id': 'req-5b5ec4c4-7938-432f-82c0-dc83f049427b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:48.004270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46717ded483e48ba8da3d4e9817f47d1', 'request_id': 'req-5b5ec4c4-7938-432f-82c0-dc83f049427b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:48.010303 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46717ded483e48ba8da3d4e9817f47d1', 'request_id': 'req-5b5ec4c4-7938-432f-82c0-dc83f049427b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:48.024720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] The token cache is being invalidate because project 46717ded483e48ba8da3d4e9817f47d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:48.025006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:48.025698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-5b5ec4c4-7938-432f-82c0-dc83f049427b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:48.027609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1031/2061] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:47 2022] DELETE /identity/v3/projects/46717ded483e48ba8da3d4e9817f47d1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:48.032757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c259e0-b210-43e7-a2b2-4b35c20429a2 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.033249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c259e0-b210-43e7-a2b2-4b35c20429a2 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.033340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c259e0-b210-43e7-a2b2-4b35c20429a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.052355 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-75c259e0-b210-43e7-a2b2-4b35c20429a2 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:48.053670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-75c259e0-b210-43e7-a2b2-4b35c20429a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:48.117596 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1031/2062] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:48.126783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f850d7-6fc3-4624-9199-a64f53354826 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.127069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f850d7-6fc3-4624-9199-a64f53354826 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.127371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f850d7-6fc3-4624-9199-a64f53354826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.146725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d6f850d7-6fc3-4624-9199-a64f53354826 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:48.147965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d6f850d7-6fc3-4624-9199-a64f53354826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:48.156445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1032/2063] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:48.160994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c259e0-b210-43e7-a2b2-4b35c20429a2 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.171394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c259e0-b210-43e7-a2b2-4b35c20429a2 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:48.176835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65caf42d-6f11-4269-a3cb-48f012c31d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:48.177493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caf42d-6f11-4269-a3cb-48f012c31d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.177769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caf42d-6f11-4269-a3cb-48f012c31d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.178042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caf42d-6f11-4269-a3cb-48f012c31d5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.178787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65caf42d-6f11-4269-a3cb-48f012c31d5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:55:48.179084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65caf42d-6f11-4269-a3cb-48f012c31d5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:48.180943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65caf42d-6f11-4269-a3cb-48f012c31d5a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:48.193828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1032/2064] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:55:48.198882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f850d7-6fc3-4624-9199-a64f53354826 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.209312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f850d7-6fc3-4624-9199-a64f53354826 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:48.214819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5de75-f9f3-4b5c-ab21-e3c01bf140f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:48.215539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5de75-f9f3-4b5c-ab21-e3c01bf140f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.215823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5de75-f9f3-4b5c-ab21-e3c01bf140f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.216101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5de75-f9f3-4b5c-ab21-e3c01bf140f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.216815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5de75-f9f3-4b5c-ab21-e3c01bf140f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:48.218760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5de75-f9f3-4b5c-ab21-e3c01bf140f6 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:48.247268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1033/2065] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:48 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:48.251685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65caf42d-6f11-4269-a3cb-48f012c31d5a admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.262122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65caf42d-6f11-4269-a3cb-48f012c31d5a admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:48.296017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:48.326837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b8baf-f71e-4f7a-aee4-4900ae15b208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:48.327554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b8baf-f71e-4f7a-aee4-4900ae15b208 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.327836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b8baf-f71e-4f7a-aee4-4900ae15b208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.328110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b8baf-f71e-4f7a-aee4-4900ae15b208 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.328820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b8baf-f71e-4f7a-aee4-4900ae15b208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:48.330834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b8baf-f71e-4f7a-aee4-4900ae15b208 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:48.353448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1033/2066] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:48 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:48.357840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5de75-f9f3-4b5c-ab21-e3c01bf140f6 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.368516 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5de75-f9f3-4b5c-ab21-e3c01bf140f6 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:48.373056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce36915-fd77-4519-a796-88fd541d968f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:48.373758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce36915-fd77-4519-a796-88fd541d968f admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.374019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce36915-fd77-4519-a796-88fd541d968f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.374307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce36915-fd77-4519-a796-88fd541d968f admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.375004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce36915-fd77-4519-a796-88fd541d968f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:48.376704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce36915-fd77-4519-a796-88fd541d968f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:48.389128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1034/2067] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:48.393676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b8baf-f71e-4f7a-aee4-4900ae15b208 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.404397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b8baf-f71e-4f7a-aee4-4900ae15b208 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:48.410023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f16ff8-84bd-4955-b3b4-be55154ee5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:48.410697 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f16ff8-84bd-4955-b3b4-be55154ee5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.410976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f16ff8-84bd-4955-b3b4-be55154ee5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.411269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f16ff8-84bd-4955-b3b4-be55154ee5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.412025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f16ff8-84bd-4955-b3b4-be55154ee5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:48.414715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f16ff8-84bd-4955-b3b4-be55154ee5a0 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:48.425931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1034/2068] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:48.431341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce36915-fd77-4519-a796-88fd541d968f admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.443030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce36915-fd77-4519-a796-88fd541d968f admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:48.448802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f6754-28ec-4d6d-b75a-ad44708e676c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:48.449742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f6754-28ec-4d6d-b75a-ad44708e676c admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.450024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f6754-28ec-4d6d-b75a-ad44708e676c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.450298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f6754-28ec-4d6d-b75a-ad44708e676c admin admin] PATH_INFO: `/v3/projects/9a92972a9311473897255cab1e08120e/users/3ed8479a844745878c3f345622e5c04a/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.465978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f6754-28ec-4d6d-b75a-ad44708e676c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a92972a9311473897255cab1e08120e, user_id=3ed8479a844745878c3f345622e5c04a, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:48.468693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f6754-28ec-4d6d-b75a-ad44708e676c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:48.484751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1035/2069] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:48 2022] PUT /identity/v3/projects/9a92972a9311473897255cab1e08120e/users/3ed8479a844745878c3f345622e5c04a/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:48.495470 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aeaab-990e-41c3-94b2-05fa01b9a6b5 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.495765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aeaab-990e-41c3-94b2-05fa01b9a6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.496042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aeaab-990e-41c3-94b2-05fa01b9a6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.528942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-2e6aeaab-990e-41c3-94b2-05fa01b9a6b5 None None] MFA Rules not processed for user `3ed8479a844745878c3f345622e5c04a`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:48.530182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2e6aeaab-990e-41c3-94b2-05fa01b9a6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:48.603877 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1035/2070] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:48.608731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f6754-28ec-4d6d-b75a-ad44708e676c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.651840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:48.679522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f6754-28ec-4d6d-b75a-ad44708e676c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:48.684778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22012b-842f-48ef-a6ab-cd1254ed4a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:48.685446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22012b-842f-48ef-a6ab-cd1254ed4a2c admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.685727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22012b-842f-48ef-a6ab-cd1254ed4a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.685997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22012b-842f-48ef-a6ab-cd1254ed4a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.686751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22012b-842f-48ef-a6ab-cd1254ed4a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:48.688641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22012b-842f-48ef-a6ab-cd1254ed4a2c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:48.723042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1036/2071] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:48 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:48.727310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aeaab-990e-41c3-94b2-05fa01b9a6b5 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.738754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aeaab-990e-41c3-94b2-05fa01b9a6b5 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:48.795047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53488c2a-20f9-4489-b2f9-65c3ff279dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:48.795807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53488c2a-20f9-4489-b2f9-65c3ff279dfa admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.796081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53488c2a-20f9-4489-b2f9-65c3ff279dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.796345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53488c2a-20f9-4489-b2f9-65c3ff279dfa admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.797078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53488c2a-20f9-4489-b2f9-65c3ff279dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:48.799482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53488c2a-20f9-4489-b2f9-65c3ff279dfa admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:48.824880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1036/2072] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:48 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:55:48.827218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22012b-842f-48ef-a6ab-cd1254ed4a2c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.838422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22012b-842f-48ef-a6ab-cd1254ed4a2c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:48.844232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83391822-621d-43ae-9330-d4d92cd96341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:48.844927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391822-621d-43ae-9330-d4d92cd96341 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.845212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391822-621d-43ae-9330-d4d92cd96341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.845486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391822-621d-43ae-9330-d4d92cd96341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.846249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83391822-621d-43ae-9330-d4d92cd96341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:48.847845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83391822-621d-43ae-9330-d4d92cd96341 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:48.860597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1037/2073] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:48.872345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53488c2a-20f9-4489-b2f9-65c3ff279dfa admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.892142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53488c2a-20f9-4489-b2f9-65c3ff279dfa admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:48.897335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17e238-e2b8-4f3e-91e8-003c69e2e50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:48.898235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17e238-e2b8-4f3e-91e8-003c69e2e50b admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:48.899344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17e238-e2b8-4f3e-91e8-003c69e2e50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:48.899344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17e238-e2b8-4f3e-91e8-003c69e2e50b admin admin] PATH_INFO: `/v3/projects/0ccd2f2b48c945289fdf796fb570ec12/users/e4e7ccea6c7048aa8609dca6c8fda26b/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:48.918174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17e238-e2b8-4f3e-91e8-003c69e2e50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccd2f2b48c945289fdf796fb570ec12, user_id=e4e7ccea6c7048aa8609dca6c8fda26b, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:48.921493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17e238-e2b8-4f3e-91e8-003c69e2e50b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:48.936538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1037/2074] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:48 2022] PUT /identity/v3/projects/0ccd2f2b48c945289fdf796fb570ec12/users/e4e7ccea6c7048aa8609dca6c8fda26b/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:48.940507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83391822-621d-43ae-9330-d4d92cd96341 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:48.951028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83391822-621d-43ae-9330-d4d92cd96341 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:49.011620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146ccbc-c4b4-47f8-ac4c-48786df5db98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:49.012348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146ccbc-c4b4-47f8-ac4c-48786df5db98 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:49.012634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146ccbc-c4b4-47f8-ac4c-48786df5db98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:49.012922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146ccbc-c4b4-47f8-ac4c-48786df5db98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:49.013636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146ccbc-c4b4-47f8-ac4c-48786df5db98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:49.015271 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146ccbc-c4b4-47f8-ac4c-48786df5db98 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:49.025203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1038/2075] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:49.029619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17e238-e2b8-4f3e-91e8-003c69e2e50b admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:49.040112 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17e238-e2b8-4f3e-91e8-003c69e2e50b admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:49.045259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d53582-b58a-42ee-bf32-ab2d7385a156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:49.046234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d53582-b58a-42ee-bf32-ab2d7385a156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:49.046678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d53582-b58a-42ee-bf32-ab2d7385a156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:49.046968 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d53582-b58a-42ee-bf32-ab2d7385a156 admin admin] PATH_INFO: `/v3/projects/0ccd2f2b48c945289fdf796fb570ec12/users/e4e7ccea6c7048aa8609dca6c8fda26b/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:49.049481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d53582-b58a-42ee-bf32-ab2d7385a156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccd2f2b48c945289fdf796fb570ec12, user_id=e4e7ccea6c7048aa8609dca6c8fda26b, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:49.052276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d53582-b58a-42ee-bf32-ab2d7385a156 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:49.058071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:49.064701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1038/2076] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:49 2022] PUT /identity/v3/projects/0ccd2f2b48c945289fdf796fb570ec12/users/e4e7ccea6c7048aa8609dca6c8fda26b/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:49.075890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3442e6e-2a60-4748-8c27-768d80c9bb3e None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:49.076397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3442e6e-2a60-4748-8c27-768d80c9bb3e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:49.076743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3442e6e-2a60-4748-8c27-768d80c9bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:49.106890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d3442e6e-2a60-4748-8c27-768d80c9bb3e None None] MFA Rules not processed for user `e4e7ccea6c7048aa8609dca6c8fda26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:49.108246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d3442e6e-2a60-4748-8c27-768d80c9bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:49.184134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1039/2077] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:49.195931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d53582-b58a-42ee-bf32-ab2d7385a156 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:49.198009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e0d53582-b58a-42ee-bf32-ab2d7385a156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:49.259656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d53582-b58a-42ee-bf32-ab2d7385a156 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:49.263614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734c83e-0fee-446e-b503-74ec58e14504 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:49.264262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734c83e-0fee-446e-b503-74ec58e14504 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:49.264547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734c83e-0fee-446e-b503-74ec58e14504 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:49.264823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734c83e-0fee-446e-b503-74ec58e14504 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:49.325057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734c83e-0fee-446e-b503-74ec58e14504 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:49.326594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734c83e-0fee-446e-b503-74ec58e14504 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:49.336345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1039/2078] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:49.952696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3442e6e-2a60-4748-8c27-768d80c9bb3e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:49.954291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d3442e6e-2a60-4748-8c27-768d80c9bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:49.989789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:50.029630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3442e6e-2a60-4748-8c27-768d80c9bb3e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:50.035285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:50.035664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:50.036652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:50.036652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] PATH_INFO: `/v3/users/5c9e99b960d240e787e4284450ad887f` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:50.037616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9e99b960d240e787e4284450ad887f)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:50.039723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:50.119228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9e99b960d240e787e4284450ad887f'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.128155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9e99b960d240e787e4284450ad887f'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.145363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9e99b960d240e787e4284450ad887f'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.146266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9e99b960d240e787e4284450ad887f'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.147099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9e99b960d240e787e4284450ad887f'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.178838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1040/2079] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:49 2022] DELETE /identity/v3/users/5c9e99b960d240e787e4284450ad887f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:50.183751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ceac6-bc1c-4971-93f6-9d90c00dd75f None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:50.184249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ceac6-bc1c-4971-93f6-9d90c00dd75f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:50.184423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ceac6-bc1c-4971-93f6-9d90c00dd75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:50.213272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-492ceac6-bc1c-4971-93f6-9d90c00dd75f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:50.214500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-492ceac6-bc1c-4971-93f6-9d90c00dd75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:50.288447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1040/2080] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:50.298919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:50.300456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:50.365697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a135-b640-4bd8-ae14-531dbd3eb77d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:50.369419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c34f2-66a9-4193-981c-c5604763b3ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:50.370043 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c34f2-66a9-4193-981c-c5604763b3ee service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:50.370324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c34f2-66a9-4193-981c-c5604763b3ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:50.370597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c34f2-66a9-4193-981c-c5604763b3ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:50.379908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c34f2-66a9-4193-981c-c5604763b3ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:50.381337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c34f2-66a9-4193-981c-c5604763b3ee service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:50.391350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1041/2081] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:50.681444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ceac6-bc1c-4971-93f6-9d90c00dd75f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:50.684826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-492ceac6-bc1c-4971-93f6-9d90c00dd75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:50.708581 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ceac6-bc1c-4971-93f6-9d90c00dd75f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:50.715386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:50.716412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:50.716838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:50.717131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] PATH_INFO: `/v3/projects/bde733786e2a48278fc6c4193c0d2adb` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:50.718532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde733786e2a48278fc6c4193c0d2adb)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:50.721152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:50.783209 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:50.785851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde733786e2a48278fc6c4193c0d2adb', 'request_id': 'req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.793799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde733786e2a48278fc6c4193c0d2adb', 'request_id': 'req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.802002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde733786e2a48278fc6c4193c0d2adb', 'request_id': 'req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.819464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] The token cache is being invalidate because project bde733786e2a48278fc6c4193c0d2adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:50.819464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.820048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.821919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1041/2082] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:50 2022] DELETE /identity/v3/projects/bde733786e2a48278fc6c4193c0d2adb => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:50.826033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c34f2-66a9-4193-981c-c5604763b3ee service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:50.837071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c34f2-66a9-4193-981c-c5604763b3ee service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:50.901894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:50.920598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:50.921125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:50.921486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:50.921770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] PATH_INFO: `/v3/users/2c142fad085f415290289dd5ef267d97` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:50.923804 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c142fad085f415290289dd5ef267d97)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:50.925787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:50.984992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c142fad085f415290289dd5ef267d97'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:50.992398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c142fad085f415290289dd5ef267d97'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:51.001626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c142fad085f415290289dd5ef267d97'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:51.002283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c142fad085f415290289dd5ef267d97'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:51.002982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d82b1985-c7fe-48a2-9ef1-7cdfc21151be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c142fad085f415290289dd5ef267d97'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:51.037350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1042/2083] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:50 2022] DELETE /identity/v3/users/2c142fad085f415290289dd5ef267d97 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:51.323627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:51.325256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:51.339406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec8d24-3765-461e-a54f-ccfb5cecdaa6 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:51.398686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:51.399586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:51.399859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:51.400130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] PATH_INFO: `/v3/projects/f317a0144b7e4c4a972281f20dc8505f` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:51.401294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f317a0144b7e4c4a972281f20dc8505f)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:51.403321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:51.453957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f317a0144b7e4c4a972281f20dc8505f', 'request_id': 'req-544547e0-3233-4f5e-848a-a18d29fbbe19'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:51.461181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f317a0144b7e4c4a972281f20dc8505f', 'request_id': 'req-544547e0-3233-4f5e-848a-a18d29fbbe19'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:51.467417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f317a0144b7e4c4a972281f20dc8505f', 'request_id': 'req-544547e0-3233-4f5e-848a-a18d29fbbe19'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:51.481410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] The token cache is being invalidate because project f317a0144b7e4c4a972281f20dc8505f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:51.481690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:51.482289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-544547e0-3233-4f5e-848a-a18d29fbbe19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:51.483886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1042/2084] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:51 2022] DELETE /identity/v3/projects/f317a0144b7e4c4a972281f20dc8505f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:51.489139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059e7d-34a7-4763-8fe4-4934f671b8de None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:51.489429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059e7d-34a7-4763-8fe4-4934f671b8de None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:51.489700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059e7d-34a7-4763-8fe4-4934f671b8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:51.508158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-9a059e7d-34a7-4763-8fe4-4934f671b8de None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:51.509356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-9a059e7d-34a7-4763-8fe4-4934f671b8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:51.569358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1043/2085] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:51.577220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2526e78-e014-42be-a9e5-e71aedb5360b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:51.577512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2526e78-e014-42be-a9e5-e71aedb5360b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:51.577794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2526e78-e014-42be-a9e5-e71aedb5360b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:51.606076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b2526e78-e014-42be-a9e5-e71aedb5360b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:51.607369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b2526e78-e014-42be-a9e5-e71aedb5360b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:51.615424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1043/2086] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:51.619964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a059e7d-34a7-4763-8fe4-4934f671b8de None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:51.630123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a059e7d-34a7-4763-8fe4-4934f671b8de None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:51.635234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5416f35-e2ac-47a2-bd23-8e0a471d1508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:51.635943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5416f35-e2ac-47a2-bd23-8e0a471d1508 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:51.636228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5416f35-e2ac-47a2-bd23-8e0a471d1508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:51.636506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5416f35-e2ac-47a2-bd23-8e0a471d1508 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:51.637281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5416f35-e2ac-47a2-bd23-8e0a471d1508 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:55:51.637589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5416f35-e2ac-47a2-bd23-8e0a471d1508 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:51.639273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5416f35-e2ac-47a2-bd23-8e0a471d1508 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:51.656857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2526e78-e014-42be-a9e5-e71aedb5360b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:51.664589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1044/2087] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:55:51.669954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2526e78-e014-42be-a9e5-e71aedb5360b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:51.674971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f48ee-efa1-4214-b6a4-e53f95ac6a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:51.675695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f48ee-efa1-4214-b6a4-e53f95ac6a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:51.675978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f48ee-efa1-4214-b6a4-e53f95ac6a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:51.676252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f48ee-efa1-4214-b6a4-e53f95ac6a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:51.676992 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f48ee-efa1-4214-b6a4-e53f95ac6a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:51.678927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f48ee-efa1-4214-b6a4-e53f95ac6a13 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:51.719083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1044/2088] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:51 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:51.726044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5416f35-e2ac-47a2-bd23-8e0a471d1508 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:51.744832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5416f35-e2ac-47a2-bd23-8e0a471d1508 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:51.811933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d78530-4d78-4aca-85f3-e998e649244c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:51.812639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d78530-4d78-4aca-85f3-e998e649244c admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:51.812895 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d78530-4d78-4aca-85f3-e998e649244c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:51.813204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d78530-4d78-4aca-85f3-e998e649244c admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:51.813856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d78530-4d78-4aca-85f3-e998e649244c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:51.815858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d78530-4d78-4aca-85f3-e998e649244c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:51.823278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:51.842638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1045/2089] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:51 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:51.846893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f48ee-efa1-4214-b6a4-e53f95ac6a13 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:51.857471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f48ee-efa1-4214-b6a4-e53f95ac6a13 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:51.863820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea852d-42ed-4eb7-9668-07314b1a03ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:51.864501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea852d-42ed-4eb7-9668-07314b1a03ba admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:51.864784 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea852d-42ed-4eb7-9668-07314b1a03ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:51.865062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea852d-42ed-4eb7-9668-07314b1a03ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:51.865810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea852d-42ed-4eb7-9668-07314b1a03ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:51.867467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea852d-42ed-4eb7-9668-07314b1a03ba admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:51.879305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1045/2090] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:51.883510 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d78530-4d78-4aca-85f3-e998e649244c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:51.899889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d78530-4d78-4aca-85f3-e998e649244c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:51.905319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb7692-8f8b-4d45-a61b-f88d71e259b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:51.906132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7692-8f8b-4d45-a61b-f88d71e259b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:51.906446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7692-8f8b-4d45-a61b-f88d71e259b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:51.906726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7692-8f8b-4d45-a61b-f88d71e259b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:51.907530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb7692-8f8b-4d45-a61b-f88d71e259b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:51.909153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb7692-8f8b-4d45-a61b-f88d71e259b6 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:51.919965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1046/2091] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:51.924452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea852d-42ed-4eb7-9668-07314b1a03ba admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:51.934688 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea852d-42ed-4eb7-9668-07314b1a03ba admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:51.939312 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca55a57-17c0-41ff-9539-3660f5c401a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:51.940212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca55a57-17c0-41ff-9539-3660f5c401a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:51.940490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca55a57-17c0-41ff-9539-3660f5c401a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:51.940764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca55a57-17c0-41ff-9539-3660f5c401a8 admin admin] PATH_INFO: `/v3/projects/0b1d9e74849846e9b80724d9a97fa0fc/users/f16b3968619e44cf930d38408283b289/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:51.959686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca55a57-17c0-41ff-9539-3660f5c401a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1d9e74849846e9b80724d9a97fa0fc, user_id=f16b3968619e44cf930d38408283b289, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:51.963941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca55a57-17c0-41ff-9539-3660f5c401a8 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:51.982238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1046/2092] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:51 2022] PUT /identity/v3/projects/0b1d9e74849846e9b80724d9a97fa0fc/users/f16b3968619e44cf930d38408283b289/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:51.993280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed4f56-84aa-4216-aa95-87ca9a7c746c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:51.993671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed4f56-84aa-4216-aa95-87ca9a7c746c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:51.993856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed4f56-84aa-4216-aa95-87ca9a7c746c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:52.021358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-bfed4f56-84aa-4216-aa95-87ca9a7c746c None None] MFA Rules not processed for user `f16b3968619e44cf930d38408283b289`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:52.022608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-bfed4f56-84aa-4216-aa95-87ca9a7c746c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:52.090560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1047/2093] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:52.102306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca55a57-17c0-41ff-9539-3660f5c401a8 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:52.104222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9ca55a57-17c0-41ff-9539-3660f5c401a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:52.110270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:52.165841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca55a57-17c0-41ff-9539-3660f5c401a8 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:52.169727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193e368-233e-478c-8e95-1f1f8e0b895f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:52.170378 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193e368-233e-478c-8e95-1f1f8e0b895f service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:52.170658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193e368-233e-478c-8e95-1f1f8e0b895f service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:52.170936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193e368-233e-478c-8e95-1f1f8e0b895f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:52.180151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193e368-233e-478c-8e95-1f1f8e0b895f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:52.181642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193e368-233e-478c-8e95-1f1f8e0b895f service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:52.191007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1047/2094] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:53.621140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed4f56-84aa-4216-aa95-87ca9a7c746c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:53.632448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed4f56-84aa-4216-aa95-87ca9a7c746c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:53.636381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac032-f6c9-4c8d-a947-7e21f75295a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:53.637059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac032-f6c9-4c8d-a947-7e21f75295a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:53.637346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac032-f6c9-4c8d-a947-7e21f75295a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:53.637782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac032-f6c9-4c8d-a947-7e21f75295a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:53.639520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cbeac032-f6c9-4c8d-a947-7e21f75295a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:53.715037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeac032-f6c9-4c8d-a947-7e21f75295a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:53.716689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeac032-f6c9-4c8d-a947-7e21f75295a7 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:53.727484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1048/2095] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:55.380594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193e368-233e-478c-8e95-1f1f8e0b895f service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:55.452626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193e368-233e-478c-8e95-1f1f8e0b895f service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:55.452626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:55.452626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:55.452626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:55.452967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] PATH_INFO: `/v3/users/f16b3968619e44cf930d38408283b289` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:55.453603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16b3968619e44cf930d38408283b289)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:55.455678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:55.521531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16b3968619e44cf930d38408283b289'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:55.531841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16b3968619e44cf930d38408283b289'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:55.532888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16b3968619e44cf930d38408283b289'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:55.542129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16b3968619e44cf930d38408283b289'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:55.558665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16b3968619e44cf930d38408283b289'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:55.567470 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1048/2096] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:55 2022] DELETE /identity/v3/users/f16b3968619e44cf930d38408283b289 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:55.572133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eac84-88c3-4d83-a557-a2f233060427 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:55.572643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eac84-88c3-4d83-a557-a2f233060427 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:55.572736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eac84-88c3-4d83-a557-a2f233060427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:55.590523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c33eac84-88c3-4d83-a557-a2f233060427 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:55.591756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c33eac84-88c3-4d83-a557-a2f233060427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:55.673302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1049/2097] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:55.689643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:55.691895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:55.744433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:55.780450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539bde5-b284-437a-9013-83d0f9437c0d admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:55.784107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10f648-7a89-4e90-a781-b006068a8a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:55.784779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10f648-7a89-4e90-a781-b006068a8a2c service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:55.785170 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10f648-7a89-4e90-a781-b006068a8a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:55.785472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10f648-7a89-4e90-a781-b006068a8a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:55.802996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10f648-7a89-4e90-a781-b006068a8a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:55.804546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10f648-7a89-4e90-a781-b006068a8a2c service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:55.816177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1049/2098] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:56.135914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eac84-88c3-4d83-a557-a2f233060427 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:56.137456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c33eac84-88c3-4d83-a557-a2f233060427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:56.145855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:56.152983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eac84-88c3-4d83-a557-a2f233060427 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:56.161155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:56.162092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.162354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.162895 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] PATH_INFO: `/v3/projects/0b1d9e74849846e9b80724d9a97fa0fc` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.164437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1d9e74849846e9b80724d9a97fa0fc)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:56.166553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:56.217589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1d9e74849846e9b80724d9a97fa0fc', 'request_id': 'req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:56.224009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1d9e74849846e9b80724d9a97fa0fc', 'request_id': 'req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:56.229513 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1d9e74849846e9b80724d9a97fa0fc', 'request_id': 'req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:56.242891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] The token cache is being invalidate because project 0b1d9e74849846e9b80724d9a97fa0fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:56.243186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:56.244396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b4f1f3bb-c97a-4f39-b811-8c6c8e47f3a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:56.245961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1050/2099] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:56 2022] DELETE /identity/v3/projects/0b1d9e74849846e9b80724d9a97fa0fc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:56.251492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea124c-28ea-4723-ac43-5a95d704dd1b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.251781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea124c-28ea-4723-ac43-5a95d704dd1b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.252058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea124c-28ea-4723-ac43-5a95d704dd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.270074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-3eea124c-28ea-4723-ac43-5a95d704dd1b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:56.271431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3eea124c-28ea-4723-ac43-5a95d704dd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:56.337803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1050/2100] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:56.347551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f9ee0-143d-49ed-9893-d3216bed5beb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.347821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f9ee0-143d-49ed-9893-d3216bed5beb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.348105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f9ee0-143d-49ed-9893-d3216bed5beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.374039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2c8f9ee0-143d-49ed-9893-d3216bed5beb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:56.375276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2c8f9ee0-143d-49ed-9893-d3216bed5beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:56.383393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1051/2101] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:56.387841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea124c-28ea-4723-ac43-5a95d704dd1b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:56.400463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea124c-28ea-4723-ac43-5a95d704dd1b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:56.405589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2285-a030-488d-97d0-73c3d4cda8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:56.406358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2285-a030-488d-97d0-73c3d4cda8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.406837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2285-a030-488d-97d0-73c3d4cda8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.407108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2285-a030-488d-97d0-73c3d4cda8e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.407872 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d2285-a030-488d-97d0-73c3d4cda8e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:55:56.408177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d2285-a030-488d-97d0-73c3d4cda8e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:56.409799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d2285-a030-488d-97d0-73c3d4cda8e4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:56.424683 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1051/2102] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:55:56.432348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f9ee0-143d-49ed-9893-d3216bed5beb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:56.445315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f9ee0-143d-49ed-9893-d3216bed5beb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:56.451435 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d431-1454-4621-9d0a-c95cbf79be3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:56.452417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d431-1454-4621-9d0a-c95cbf79be3a admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.452696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d431-1454-4621-9d0a-c95cbf79be3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.452991 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d431-1454-4621-9d0a-c95cbf79be3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.453658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373d431-1454-4621-9d0a-c95cbf79be3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:56.455597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373d431-1454-4621-9d0a-c95cbf79be3a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:56.489738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1052/2103] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:55:56 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:56.491382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2285-a030-488d-97d0-73c3d4cda8e4 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:56.508568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2285-a030-488d-97d0-73c3d4cda8e4 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:56.570567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747600c5-9f48-4fc4-bfe3-44d6faf2f835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:56.571165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747600c5-9f48-4fc4-bfe3-44d6faf2f835 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.571475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747600c5-9f48-4fc4-bfe3-44d6faf2f835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.571775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747600c5-9f48-4fc4-bfe3-44d6faf2f835 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.572465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747600c5-9f48-4fc4-bfe3-44d6faf2f835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:56.574420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747600c5-9f48-4fc4-bfe3-44d6faf2f835 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:56.606008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1052/2104] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:55:56 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:55:56.607640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d431-1454-4621-9d0a-c95cbf79be3a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:56.622161 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d431-1454-4621-9d0a-c95cbf79be3a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:56.627687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999392d-5dc1-4c58-a910-92cc656e1942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:56.628381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999392d-5dc1-4c58-a910-92cc656e1942 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.628662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999392d-5dc1-4c58-a910-92cc656e1942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.628958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999392d-5dc1-4c58-a910-92cc656e1942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.629671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999392d-5dc1-4c58-a910-92cc656e1942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:56.631291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999392d-5dc1-4c58-a910-92cc656e1942 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:56.642872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1053/2105] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:55:56.647022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747600c5-9f48-4fc4-bfe3-44d6faf2f835 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:56.657035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747600c5-9f48-4fc4-bfe3-44d6faf2f835 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:56.661854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631761f7-d9fe-4265-b19b-d9a4236b97b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:56.662566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631761f7-d9fe-4265-b19b-d9a4236b97b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.662842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631761f7-d9fe-4265-b19b-d9a4236b97b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.663099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631761f7-d9fe-4265-b19b-d9a4236b97b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.663882 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631761f7-d9fe-4265-b19b-d9a4236b97b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:56.665502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631761f7-d9fe-4265-b19b-d9a4236b97b3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:56.676018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1053/2106] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:55:56.680112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999392d-5dc1-4c58-a910-92cc656e1942 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:56.690546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999392d-5dc1-4c58-a910-92cc656e1942 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:56.695388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc92cab-b33d-43cc-a0ef-2f14aeeb3c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:56.696311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc92cab-b33d-43cc-a0ef-2f14aeeb3c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.696592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc92cab-b33d-43cc-a0ef-2f14aeeb3c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.696867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc92cab-b33d-43cc-a0ef-2f14aeeb3c25 admin admin] PATH_INFO: `/v3/projects/ee9a610826bc47b395a26f07ad375965/users/9dd9a1603e2340eba37ae8699f40b9f6/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.712079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc92cab-b33d-43cc-a0ef-2f14aeeb3c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9a610826bc47b395a26f07ad375965, user_id=9dd9a1603e2340eba37ae8699f40b9f6, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:56.714749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc92cab-b33d-43cc-a0ef-2f14aeeb3c25 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:56.730415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1054/2107] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:55:56 2022] PUT /identity/v3/projects/ee9a610826bc47b395a26f07ad375965/users/9dd9a1603e2340eba37ae8699f40b9f6/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:56.735819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3b418-7d84-4827-a362-367284fcd0aa None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.736197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3b418-7d84-4827-a362-367284fcd0aa None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.736474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3b418-7d84-4827-a362-367284fcd0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.755957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-f0b3b418-7d84-4827-a362-367284fcd0aa None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:56.757203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f0b3b418-7d84-4827-a362-367284fcd0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:56.791616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:56.825525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1054/2108] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:56.841306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc92cab-b33d-43cc-a0ef-2f14aeeb3c25 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:56.843602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8bc92cab-b33d-43cc-a0ef-2f14aeeb3c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:56.909961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc92cab-b33d-43cc-a0ef-2f14aeeb3c25 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:56.913680 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45b0e9-f887-4f2b-a877-e0f41cebeed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:56.914365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45b0e9-f887-4f2b-a877-e0f41cebeed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:56.914671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45b0e9-f887-4f2b-a877-e0f41cebeed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:56.914943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45b0e9-f887-4f2b-a877-e0f41cebeed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:56.925350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45b0e9-f887-4f2b-a877-e0f41cebeed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:56.926944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45b0e9-f887-4f2b-a877-e0f41cebeed9 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:56.937091 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1055/2109] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:57.577783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d556-2c6c-4034-ab90-6228394730a3 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:57.578273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d556-2c6c-4034-ab90-6228394730a3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:57.578273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d556-2c6c-4034-ab90-6228394730a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:57.642080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-be42d556-2c6c-4034-ab90-6228394730a3 None None] MFA Rules not processed for user `9dd9a1603e2340eba37ae8699f40b9f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:57.643566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-be42d556-2c6c-4034-ab90-6228394730a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:57.735809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1055/2110] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:55:57.744297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45b0e9-f887-4f2b-a877-e0f41cebeed9 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:57.746641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fe45b0e9-f887-4f2b-a877-e0f41cebeed9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:57.776794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:57.809037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45b0e9-f887-4f2b-a877-e0f41cebeed9 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:57.812807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0d704-31a5-4f5f-8145-5d6a27b03a2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:57.813463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0d704-31a5-4f5f-8145-5d6a27b03a2d service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:57.813743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0d704-31a5-4f5f-8145-5d6a27b03a2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:57.814012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0d704-31a5-4f5f-8145-5d6a27b03a2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:57.822371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0d704-31a5-4f5f-8145-5d6a27b03a2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:57.823855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0d704-31a5-4f5f-8145-5d6a27b03a2d service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:57.832914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1056/2111] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 05:55:58.787412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42d556-2c6c-4034-ab90-6228394730a3 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:58.789130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-be42d556-2c6c-4034-ab90-6228394730a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:58.804384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42d556-2c6c-4034-ab90-6228394730a3 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:58.809251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:58.810145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:58.810427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:58.810704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] PATH_INFO: `/v3/users/3ed8479a844745878c3f345622e5c04a` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:58.811952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ed8479a844745878c3f345622e5c04a)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:58.813894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:58.869226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ed8479a844745878c3f345622e5c04a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:58.877408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ed8479a844745878c3f345622e5c04a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:58.878296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ed8479a844745878c3f345622e5c04a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:58.886512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ed8479a844745878c3f345622e5c04a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:58.899181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ed8479a844745878c3f345622e5c04a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:58.905396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1056/2112] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:58 2022] DELETE /identity/v3/users/3ed8479a844745878c3f345622e5c04a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:58.910370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f758c83-2de2-4b6b-8552-81d7d0ce489b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:58.910889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f758c83-2de2-4b6b-8552-81d7d0ce489b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:58.911018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f758c83-2de2-4b6b-8552-81d7d0ce489b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:58.929478 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4f758c83-2de2-4b6b-8552-81d7d0ce489b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:58.930662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4f758c83-2de2-4b6b-8552-81d7d0ce489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:58.990541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1057/2113] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:59.000908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:59.002416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:59.043594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:59.063229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bef3a-173c-4fcf-aa7c-88a58a70e05e admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:59.067082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f0c60-da2f-461f-a0f9-c1c7d7268a97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:59.067768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f0c60-da2f-461f-a0f9-c1c7d7268a97 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:59.068071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f0c60-da2f-461f-a0f9-c1c7d7268a97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:59.068347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f0c60-da2f-461f-a0f9-c1c7d7268a97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:59.077692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f0c60-da2f-461f-a0f9-c1c7d7268a97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:59.079150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f0c60-da2f-461f-a0f9-c1c7d7268a97 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:59.088685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1057/2114] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:55:59.224228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce6246-a637-4d20-9947-8c51aae341ad None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:59.224573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce6246-a637-4d20-9947-8c51aae341ad None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:59.224906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce6246-a637-4d20-9947-8c51aae341ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:59.244165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-f3ce6246-a637-4d20-9947-8c51aae341ad None None] MFA Rules not processed for user `9dd9a1603e2340eba37ae8699f40b9f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:55:59.245389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f3ce6246-a637-4d20-9947-8c51aae341ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:59.315615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1058/2115] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:55:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:55:59.535298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f0c60-da2f-461f-a0f9-c1c7d7268a97 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:59.536783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-630f0c60-da2f-461f-a0f9-c1c7d7268a97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:55:59.550783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f0c60-da2f-461f-a0f9-c1c7d7268a97 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:59.565732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:59.566668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:59.566926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:59.567195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] PATH_INFO: `/v3/projects/9a92972a9311473897255cab1e08120e` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:59.568615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a92972a9311473897255cab1e08120e)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:59.570548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:59.621490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a92972a9311473897255cab1e08120e', 'request_id': 'req-b17a8194-a793-4b33-898d-d63265435f7c'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:59.628140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a92972a9311473897255cab1e08120e', 'request_id': 'req-b17a8194-a793-4b33-898d-d63265435f7c'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:59.634601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a92972a9311473897255cab1e08120e', 'request_id': 'req-b17a8194-a793-4b33-898d-d63265435f7c'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:59.649893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] The token cache is being invalidate because project 9a92972a9311473897255cab1e08120e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:55:59.650175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:59.650808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:59.652396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1058/2116] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:55:59 2022] DELETE /identity/v3/projects/9a92972a9311473897255cab1e08120e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:55:59.656553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce6246-a637-4d20-9947-8c51aae341ad None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:55:59.667527 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce6246-a637-4d20-9947-8c51aae341ad None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:55:59.725431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:55:59.726326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:55:59.726624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:55:59.726875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] PATH_INFO: `/v3/users/e4e7ccea6c7048aa8609dca6c8fda26b` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:55:59.728134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e7ccea6c7048aa8609dca6c8fda26b)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:55:59.730356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:55:59.744467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:55:59.780991 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e7ccea6c7048aa8609dca6c8fda26b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:59.788546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e7ccea6c7048aa8609dca6c8fda26b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:59.798860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e7ccea6c7048aa8609dca6c8fda26b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:59.799632 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e7ccea6c7048aa8609dca6c8fda26b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:59.800233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-17f93fa4-d059-4a4f-bf12-c8d76857a0f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e7ccea6c7048aa8609dca6c8fda26b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:55:59.818093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1059/2117] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:55:59 2022] DELETE /identity/v3/users/e4e7ccea6c7048aa8609dca6c8fda26b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:56:00.169257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:00.170890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:00.191142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a8194-a793-4b33-898d-d63265435f7c admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:00.263693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:00.264574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:00.264855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:00.265213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] PATH_INFO: `/v3/projects/0ccd2f2b48c945289fdf796fb570ec12` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:00.266430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ccd2f2b48c945289fdf796fb570ec12)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:00.268417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:00.317383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ccd2f2b48c945289fdf796fb570ec12', 'request_id': 'req-db4c2931-ef5e-49e2-8513-f159ebada8a8'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:00.323651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ccd2f2b48c945289fdf796fb570ec12', 'request_id': 'req-db4c2931-ef5e-49e2-8513-f159ebada8a8'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:00.329849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ccd2f2b48c945289fdf796fb570ec12', 'request_id': 'req-db4c2931-ef5e-49e2-8513-f159ebada8a8'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:00.331595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:00.345484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] The token cache is being invalidate because project 0ccd2f2b48c945289fdf796fb570ec12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:56:00.345879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:00.346665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-db4c2931-ef5e-49e2-8513-f159ebada8a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:00.348266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1059/2118] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:56:00 2022] DELETE /identity/v3/projects/0ccd2f2b48c945289fdf796fb570ec12 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:56:00.354382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0effdc1-cfa8-4585-a82f-42d9190e900a None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:00.354865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0effdc1-cfa8-4585-a82f-42d9190e900a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:00.354984 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0effdc1-cfa8-4585-a82f-42d9190e900a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:00.373760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-f0effdc1-cfa8-4585-a82f-42d9190e900a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:00.375035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f0effdc1-cfa8-4585-a82f-42d9190e900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:00.462081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1060/2119] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:56:00.470242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51cca7-8d92-4742-9346-8f602c44438f None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:00.470526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51cca7-8d92-4742-9346-8f602c44438f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:00.470797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51cca7-8d92-4742-9346-8f602c44438f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:00.490613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-9a51cca7-8d92-4742-9346-8f602c44438f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:00.491937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9a51cca7-8d92-4742-9346-8f602c44438f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:00.500155 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1060/2120] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:00.506602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0effdc1-cfa8-4585-a82f-42d9190e900a None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:00.519493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0effdc1-cfa8-4585-a82f-42d9190e900a None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:00.524543 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95355210-8657-4b09-a902-0e34a615f903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:00.525229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95355210-8657-4b09-a902-0e34a615f903 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:00.525699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95355210-8657-4b09-a902-0e34a615f903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:00.526131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95355210-8657-4b09-a902-0e34a615f903 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:00.526873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95355210-8657-4b09-a902-0e34a615f903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:56:00.527402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95355210-8657-4b09-a902-0e34a615f903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:00.529172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95355210-8657-4b09-a902-0e34a615f903 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:00.542329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1061/2121] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:56:00.547745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51cca7-8d92-4742-9346-8f602c44438f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:00.559122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51cca7-8d92-4742-9346-8f602c44438f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:00.564244 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb29e3b-6270-4d30-873a-18ddc52667ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:00.564981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb29e3b-6270-4d30-873a-18ddc52667ac admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:00.565266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb29e3b-6270-4d30-873a-18ddc52667ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:00.566279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb29e3b-6270-4d30-873a-18ddc52667ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:00.567005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb29e3b-6270-4d30-873a-18ddc52667ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:00.569362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb29e3b-6270-4d30-873a-18ddc52667ac admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:00.603781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1061/2122] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:56:00 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:56:00.608318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95355210-8657-4b09-a902-0e34a615f903 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:00.619269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95355210-8657-4b09-a902-0e34a615f903 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:00.681737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa827b-acad-475f-80e5-ed8894961ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:00.682420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa827b-acad-475f-80e5-ed8894961ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:00.682705 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa827b-acad-475f-80e5-ed8894961ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:00.682973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa827b-acad-475f-80e5-ed8894961ceb admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:00.683727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa827b-acad-475f-80e5-ed8894961ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:00.685667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa827b-acad-475f-80e5-ed8894961ceb admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:00.710397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1062/2123] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:56:00 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:56:00.714652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb29e3b-6270-4d30-873a-18ddc52667ac admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:00.725561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb29e3b-6270-4d30-873a-18ddc52667ac admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:00.730832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c73202-b6a2-438e-b071-009a20a030f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:00.731629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c73202-b6a2-438e-b071-009a20a030f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:00.732057 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c73202-b6a2-438e-b071-009a20a030f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:00.732323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c73202-b6a2-438e-b071-009a20a030f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:00.733072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c73202-b6a2-438e-b071-009a20a030f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:00.734730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c73202-b6a2-438e-b071-009a20a030f6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:00.745993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1062/2124] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:56:00.750262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa827b-acad-475f-80e5-ed8894961ceb admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:00.760455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa827b-acad-475f-80e5-ed8894961ceb admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:00.765630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2f4cc-c19c-4152-8a5f-6b698da67d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:00.766293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2f4cc-c19c-4152-8a5f-6b698da67d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:00.766567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2f4cc-c19c-4152-8a5f-6b698da67d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:00.766864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2f4cc-c19c-4152-8a5f-6b698da67d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:00.767760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2f4cc-c19c-4152-8a5f-6b698da67d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:00.769366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2f4cc-c19c-4152-8a5f-6b698da67d97 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:00.771414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:00.782015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1063/2125] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:56:00.785742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c73202-b6a2-438e-b071-009a20a030f6 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:00.796011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c73202-b6a2-438e-b071-009a20a030f6 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:00.802225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4e736-f76f-4456-ba7e-8a23461cb86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:00.803149 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4e736-f76f-4456-ba7e-8a23461cb86b admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:00.803451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4e736-f76f-4456-ba7e-8a23461cb86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:00.803853 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4e736-f76f-4456-ba7e-8a23461cb86b admin admin] PATH_INFO: `/v3/projects/9b45aee732d74ce98573686d0c99b048/users/d9fd428284ac4760a1c46fc2b68d1892/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:00.818964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4e736-f76f-4456-ba7e-8a23461cb86b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b45aee732d74ce98573686d0c99b048, user_id=d9fd428284ac4760a1c46fc2b68d1892, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:00.821645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4e736-f76f-4456-ba7e-8a23461cb86b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:00.837668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1063/2126] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:56:00 2022] PUT /identity/v3/projects/9b45aee732d74ce98573686d0c99b048/users/d9fd428284ac4760a1c46fc2b68d1892/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:56:00.849212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5cf05-d3c0-43e9-8a53-1c5d8524ba81 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:00.849502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5cf05-d3c0-43e9-8a53-1c5d8524ba81 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:00.849782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5cf05-d3c0-43e9-8a53-1c5d8524ba81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:00.877436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-7da5cf05-d3c0-43e9-8a53-1c5d8524ba81 None None] MFA Rules not processed for user `d9fd428284ac4760a1c46fc2b68d1892`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:00.878670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7da5cf05-d3c0-43e9-8a53-1c5d8524ba81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:00.945776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1064/2127] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:56:00.957295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4e736-f76f-4456-ba7e-8a23461cb86b admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:00.959092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d9d4e736-f76f-4456-ba7e-8a23461cb86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:01.021808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4e736-f76f-4456-ba7e-8a23461cb86b admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:01.029762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fa639-5cf6-4a77-bae5-e4a9b50e6332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:01.031887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fa639-5cf6-4a77-bae5-e4a9b50e6332 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:01.031887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fa639-5cf6-4a77-bae5-e4a9b50e6332 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:01.031887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fa639-5cf6-4a77-bae5-e4a9b50e6332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:01.041519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fa639-5cf6-4a77-bae5-e4a9b50e6332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:01.043743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fa639-5cf6-4a77-bae5-e4a9b50e6332 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:01.056346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1064/2128] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:56:00 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:56:05.679936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469c4c-5a4d-4338-8825-e3c37bd97ced None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:05.680693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469c4c-5a4d-4338-8825-e3c37bd97ced None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:05.681070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469c4c-5a4d-4338-8825-e3c37bd97ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:05.704953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c2469c4c-5a4d-4338-8825-e3c37bd97ced None None] MFA Rules not processed for user `e367c76dfa7f4d2791e60ce148f352c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:05.706178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c2469c4c-5a4d-4338-8825-e3c37bd97ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:05.764090 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1065/2129] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:11.051782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=55 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:15.122169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04c489-06e2-4463-b482-438b02546906 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:15.122805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04c489-06e2-4463-b482-438b02546906 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:15.122805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04c489-06e2-4463-b482-438b02546906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:15.144686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ad04c489-06e2-4463-b482-438b02546906 None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:15.146440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ad04c489-06e2-4463-b482-438b02546906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:15.233716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1065/2130] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 05:56:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:15.295959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469c4c-5a4d-4338-8825-e3c37bd97ced None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:15.298530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c2469c4c-5a4d-4338-8825-e3c37bd97ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:15.434889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469c4c-5a4d-4338-8825-e3c37bd97ced None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:15.452057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0875a-054d-448e-90bf-5766a9ba6016 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:15.452747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0875a-054d-448e-90bf-5766a9ba6016 service swift] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:15.453036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0875a-054d-448e-90bf-5766a9ba6016 service swift] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:15.453321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0875a-054d-448e-90bf-5766a9ba6016 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:15.467744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0875a-054d-448e-90bf-5766a9ba6016 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:15.469365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0875a-054d-448e-90bf-5766a9ba6016 service swift] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:15.491896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1066/2131] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 05:56:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 05:56:18.617578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04c489-06e2-4463-b482-438b02546906 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:18.618988 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ad04c489-06e2-4463-b482-438b02546906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:18.730494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04c489-06e2-4463-b482-438b02546906 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:18.738369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:18.739869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:18.740153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:18.740429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] PATH_INFO: `/v3/users/b02288fe1cbd4328b0e567b9c6832059` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:18.741740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b02288fe1cbd4328b0e567b9c6832059)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:18.743855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:18.872770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b02288fe1cbd4328b0e567b9c6832059'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:18.885332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b02288fe1cbd4328b0e567b9c6832059'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:18.889564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b02288fe1cbd4328b0e567b9c6832059'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:18.898911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b02288fe1cbd4328b0e567b9c6832059'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:18.918508 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b02288fe1cbd4328b0e567b9c6832059'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:18.929470 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1066/2132] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:56:18 2022] DELETE /identity/v3/users/b02288fe1cbd4328b0e567b9c6832059 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:56:18.931760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e64853-a96a-4a03-9e2f-c307b5398e1d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:18.932751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e64853-a96a-4a03-9e2f-c307b5398e1d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:18.933534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e64853-a96a-4a03-9e2f-c307b5398e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:18.952679 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-77e64853-a96a-4a03-9e2f-c307b5398e1d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:18.954568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-77e64853-a96a-4a03-9e2f-c307b5398e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:18.995921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:19.053983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1067/2133] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:19.059978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:19.062093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:19.120894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:19.175695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa507888-8457-4f18-a4b7-f538e5153f91 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:19.193152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d5fe8-5a91-4f40-a1a5-7b60b6f49a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:19.193152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d5fe8-5a91-4f40-a1a5-7b60b6f49a3e service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:19.193152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d5fe8-5a91-4f40-a1a5-7b60b6f49a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:19.193152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d5fe8-5a91-4f40-a1a5-7b60b6f49a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:19.205451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d5fe8-5a91-4f40-a1a5-7b60b6f49a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:19.207214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d5fe8-5a91-4f40-a1a5-7b60b6f49a3e service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:19.229406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1067/2134] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:56:19.752662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e64853-a96a-4a03-9e2f-c307b5398e1d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:19.754813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-77e64853-a96a-4a03-9e2f-c307b5398e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:19.778745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e64853-a96a-4a03-9e2f-c307b5398e1d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:19.790143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:19.791412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:19.791866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:19.792862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] PATH_INFO: `/v3/projects/471f30df1e0c43aab849b9c010009142` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:19.797577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=471f30df1e0c43aab849b9c010009142)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:19.799880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:19.876695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '471f30df1e0c43aab849b9c010009142', 'request_id': 'req-b46d5c60-474c-43e1-9e38-e7f8df720882'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:19.887438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '471f30df1e0c43aab849b9c010009142', 'request_id': 'req-b46d5c60-474c-43e1-9e38-e7f8df720882'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:19.896365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '471f30df1e0c43aab849b9c010009142', 'request_id': 'req-b46d5c60-474c-43e1-9e38-e7f8df720882'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:19.923861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] The token cache is being invalidate because project 471f30df1e0c43aab849b9c010009142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:56:19.924293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:19.924994 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46d5c60-474c-43e1-9e38-e7f8df720882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:19.929203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1068/2135] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:56:19 2022] DELETE /identity/v3/projects/471f30df1e0c43aab849b9c010009142 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:56:19.933166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149de4dd-9e30-455c-abac-0f79dd2c7d4f None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:19.933698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149de4dd-9e30-455c-abac-0f79dd2c7d4f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:19.934669 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149de4dd-9e30-455c-abac-0f79dd2c7d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:19.956674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-149de4dd-9e30-455c-abac-0f79dd2c7d4f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:19.957843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-149de4dd-9e30-455c-abac-0f79dd2c7d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:20.051300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1068/2136] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:56:20.058998 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380c250-4ee8-4b50-a9fc-a3ee9c3c41b4 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:20.059907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380c250-4ee8-4b50-a9fc-a3ee9c3c41b4 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:20.060174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380c250-4ee8-4b50-a9fc-a3ee9c3c41b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:20.084659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a380c250-4ee8-4b50-a9fc-a3ee9c3c41b4 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:20.086849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a380c250-4ee8-4b50-a9fc-a3ee9c3c41b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:20.099893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1069/2137] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:20.100729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149de4dd-9e30-455c-abac-0f79dd2c7d4f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:20.115308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149de4dd-9e30-455c-abac-0f79dd2c7d4f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:20.120663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4e62c-a106-45e5-9fa7-7a99e7ff34ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:20.121415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4e62c-a106-45e5-9fa7-7a99e7ff34ab admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:20.121765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4e62c-a106-45e5-9fa7-7a99e7ff34ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:20.122104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4e62c-a106-45e5-9fa7-7a99e7ff34ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:20.122911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4e62c-a106-45e5-9fa7-7a99e7ff34ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:56:20.123323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4e62c-a106-45e5-9fa7-7a99e7ff34ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:20.125017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4e62c-a106-45e5-9fa7-7a99e7ff34ab admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:20.141582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1069/2138] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 05:56:20.146071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380c250-4ee8-4b50-a9fc-a3ee9c3c41b4 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:20.155933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380c250-4ee8-4b50-a9fc-a3ee9c3c41b4 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:20.161665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d93f5-510b-4fbe-b78c-f96c74af3d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:20.162335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d93f5-510b-4fbe-b78c-f96c74af3d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:20.162942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d93f5-510b-4fbe-b78c-f96c74af3d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:20.163424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d93f5-510b-4fbe-b78c-f96c74af3d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:20.164273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d93f5-510b-4fbe-b78c-f96c74af3d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:20.166269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d93f5-510b-4fbe-b78c-f96c74af3d14 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:20.211283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1070/2139] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:56:20 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:56:20.215293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4e62c-a106-45e5-9fa7-7a99e7ff34ab admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:20.226546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4e62c-a106-45e5-9fa7-7a99e7ff34ab admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:20.378334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ab3e3-c6ab-406f-a696-eae74cc56f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:20.379396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ab3e3-c6ab-406f-a696-eae74cc56f74 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:20.381102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ab3e3-c6ab-406f-a696-eae74cc56f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:20.381593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ab3e3-c6ab-406f-a696-eae74cc56f74 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:20.384358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ab3e3-c6ab-406f-a696-eae74cc56f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:20.386534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ab3e3-c6ab-406f-a696-eae74cc56f74 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:20.432686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1070/2140] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:56:20 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:56:20.438015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d93f5-510b-4fbe-b78c-f96c74af3d14 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:20.460572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d93f5-510b-4fbe-b78c-f96c74af3d14 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:20.473766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a91bb-30d6-4e2d-93ab-56fad1526af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:20.475902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a91bb-30d6-4e2d-93ab-56fad1526af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:20.476903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a91bb-30d6-4e2d-93ab-56fad1526af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:20.477449 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a91bb-30d6-4e2d-93ab-56fad1526af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:20.478553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a91bb-30d6-4e2d-93ab-56fad1526af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:20.482839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a91bb-30d6-4e2d-93ab-56fad1526af1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:20.524922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1071/2141] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:56:20.528929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ab3e3-c6ab-406f-a696-eae74cc56f74 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:20.541113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ab3e3-c6ab-406f-a696-eae74cc56f74 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:20.555442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84654294-618d-4b12-a282-aa2888fe9c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:20.556398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84654294-618d-4b12-a282-aa2888fe9c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:20.556826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84654294-618d-4b12-a282-aa2888fe9c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:20.557271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84654294-618d-4b12-a282-aa2888fe9c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:20.558159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84654294-618d-4b12-a282-aa2888fe9c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:20.560249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84654294-618d-4b12-a282-aa2888fe9c7b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:20.584009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1071/2142] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:56:20.587362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a91bb-30d6-4e2d-93ab-56fad1526af1 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:20.604249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a91bb-30d6-4e2d-93ab-56fad1526af1 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:20.616556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f90612-3239-424a-a1ed-a2535b35d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:20.619494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f90612-3239-424a-a1ed-a2535b35d1a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:20.620672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f90612-3239-424a-a1ed-a2535b35d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:20.621114 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f90612-3239-424a-a1ed-a2535b35d1a3 admin admin] PATH_INFO: `/v3/projects/b3ee0596be3b4648bd859293e02b1944/users/e78b454f9d0b4e99ae0425cd596f02c0/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:20.648235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f90612-3239-424a-a1ed-a2535b35d1a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ee0596be3b4648bd859293e02b1944, user_id=e78b454f9d0b4e99ae0425cd596f02c0, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:20.651417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f90612-3239-424a-a1ed-a2535b35d1a3 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:20.704293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1072/2143] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:56:20 2022] PUT /identity/v3/projects/b3ee0596be3b4648bd859293e02b1944/users/e78b454f9d0b4e99ae0425cd596f02c0/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 05:56:20.708993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a8420-a410-4286-9905-391cb97f853a None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:20.709277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a8420-a410-4286-9905-391cb97f853a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:20.709556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a8420-a410-4286-9905-391cb97f853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:20.772971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-f67a8420-a410-4286-9905-391cb97f853a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:20.774247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f67a8420-a410-4286-9905-391cb97f853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:20.872714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:20.942296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1072/2144] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:20.954882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f90612-3239-424a-a1ed-a2535b35d1a3 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:20.957206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-68f90612-3239-424a-a1ed-a2535b35d1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:21.097750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:21.138747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f90612-3239-424a-a1ed-a2535b35d1a3 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:21.157343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a5001-1f1b-4885-ad73-280ec2e2dc47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:21.159004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a5001-1f1b-4885-ad73-280ec2e2dc47 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:21.159603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a5001-1f1b-4885-ad73-280ec2e2dc47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:21.160195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a5001-1f1b-4885-ad73-280ec2e2dc47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:21.178966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a5001-1f1b-4885-ad73-280ec2e2dc47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:21.180834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a5001-1f1b-4885-ad73-280ec2e2dc47 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:21.192428 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1073/2145] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:56:22.311693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c05b0-6c98-4235-b559-e0e71e2b61ec None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:22.312737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c05b0-6c98-4235-b559-e0e71e2b61ec None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:22.313235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c05b0-6c98-4235-b559-e0e71e2b61ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:22.365627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-fa9c05b0-6c98-4235-b559-e0e71e2b61ec None None] MFA Rules not processed for user `e78b454f9d0b4e99ae0425cd596f02c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:22.367014 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-fa9c05b0-6c98-4235-b559-e0e71e2b61ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:22.500914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1073/2146] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:22.517606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a5001-1f1b-4885-ad73-280ec2e2dc47 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:22.519746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d23a5001-1f1b-4885-ad73-280ec2e2dc47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:22.663756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a5001-1f1b-4885-ad73-280ec2e2dc47 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:22.673720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c3278-77bd-43a9-af9e-257431d9d190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:22.675495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c3278-77bd-43a9-af9e-257431d9d190 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:22.676284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c3278-77bd-43a9-af9e-257431d9d190 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:22.676567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c3278-77bd-43a9-af9e-257431d9d190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:22.691564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c3278-77bd-43a9-af9e-257431d9d190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:22.694877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c3278-77bd-43a9-af9e-257431d9d190 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:22.714391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1074/2147] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:56:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:56:24.412850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db1ee9-a84c-4b4f-86e2-9398334e8d82 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:24.413466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db1ee9-a84c-4b4f-86e2-9398334e8d82 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:24.413466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db1ee9-a84c-4b4f-86e2-9398334e8d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:24.437929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-11db1ee9-a84c-4b4f-86e2-9398334e8d82 None None] MFA Rules not processed for user `e78b454f9d0b4e99ae0425cd596f02c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:24.440154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-11db1ee9-a84c-4b4f-86e2-9398334e8d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:24.454424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1074/2148] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:30.300241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b1648-9fa9-4961-843a-e80cd6757f1b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:30.300843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b1648-9fa9-4961-843a-e80cd6757f1b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:30.300843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b1648-9fa9-4961-843a-e80cd6757f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:30.336438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-062b1648-9fa9-4961-843a-e80cd6757f1b None None] MFA Rules not processed for user `9dd9a1603e2340eba37ae8699f40b9f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:30.338226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-062b1648-9fa9-4961-843a-e80cd6757f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:30.457758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1075/2149] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:34.431746 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=47 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:35.757925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd57e1-e957-4428-b2ec-9aac32f42c6d None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:35.758576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd57e1-e957-4428-b2ec-9aac32f42c6d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:35.758576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd57e1-e957-4428-b2ec-9aac32f42c6d None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:35.760749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1075/2150] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 05:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:56:35.768914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b67659-740b-4278-b6b4-c483d1ef5dd4 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:35.769390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b67659-740b-4278-b6b4-c483d1ef5dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:35.769849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b67659-740b-4278-b6b4-c483d1ef5dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:35.792002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-56b67659-740b-4278-b6b4-c483d1ef5dd4 None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:35.793430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-56b67659-740b-4278-b6b4-c483d1ef5dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:35.899320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:35.947345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1076/2151] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 05:56:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:35.963142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd57e1-e957-4428-b2ec-9aac32f42c6d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:35.963142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-08bd57e1-e957-4428-b2ec-9aac32f42c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:36.038377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd57e1-e957-4428-b2ec-9aac32f42c6d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:36.043521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b377680-073b-4863-8a30-a62660a9ee09 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab33473d40094ec681e538cde250ec2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:36.043778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b377680-073b-4863-8a30-a62660a9ee09 service glance] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:36.044059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b377680-073b-4863-8a30-a62660a9ee09 service glance] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:36.044333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b377680-073b-4863-8a30-a62660a9ee09 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:36.059653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b377680-073b-4863-8a30-a62660a9ee09 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:36.061703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b377680-073b-4863-8a30-a62660a9ee09 service glance] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:36.086697 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1076/2152] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:56:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:56:36.155204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71360c4-591e-4912-8f99-9d96082c2d14 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:36.156085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71360c4-591e-4912-8f99-9d96082c2d14 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:36.156085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71360c4-591e-4912-8f99-9d96082c2d14 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:36.157958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1077/2153] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 05:56:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:56:36.165746 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060dd6f-3b2e-4838-8296-d2a390614713 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:36.166285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060dd6f-3b2e-4838-8296-d2a390614713 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:36.166726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060dd6f-3b2e-4838-8296-d2a390614713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:36.224910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d060dd6f-3b2e-4838-8296-d2a390614713 None None] MFA Rules not processed for user `817b1a026f4b443daea514338ad8531a`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:36.227101 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d060dd6f-3b2e-4838-8296-d2a390614713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:36.250025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1077/2154] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 05:56:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:56:36.286698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71360c4-591e-4912-8f99-9d96082c2d14 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:36.303720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71360c4-591e-4912-8f99-9d96082c2d14 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:36.308982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced72276-cb33-4bf2-852d-b3df69c86a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:36.309872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced72276-cb33-4bf2-852d-b3df69c86a2a service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:36.310222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced72276-cb33-4bf2-852d-b3df69c86a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:36.310738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced72276-cb33-4bf2-852d-b3df69c86a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:36.324843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced72276-cb33-4bf2-852d-b3df69c86a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:36.326510 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced72276-cb33-4bf2-852d-b3df69c86a2a service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:36.337206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1078/2155] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:56:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:56:36.547998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075c3d-1d5b-4e40-a128-4e4cefdad502 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:36.548226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075c3d-1d5b-4e40-a128-4e4cefdad502 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:36.548499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075c3d-1d5b-4e40-a128-4e4cefdad502 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:36.549887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1078/2156] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 05:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 05:56:36.555827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76306fff-7b57-4e7d-b2e3-4be410b8d5b8 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:36.556826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76306fff-7b57-4e7d-b2e3-4be410b8d5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:36.557181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76306fff-7b57-4e7d-b2e3-4be410b8d5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:36.579513 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-76306fff-7b57-4e7d-b2e3-4be410b8d5b8 None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:36.580692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-76306fff-7b57-4e7d-b2e3-4be410b8d5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:36.590696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1079/2157] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 05:56:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:36.602097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075c3d-1d5b-4e40-a128-4e4cefdad502 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:36.604077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3a075c3d-1d5b-4e40-a128-4e4cefdad502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:36.617722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075c3d-1d5b-4e40-a128-4e4cefdad502 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:36.621281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170833bb-87f2-478b-b2f0-0c1224465289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:36.621926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170833bb-87f2-478b-b2f0-0c1224465289 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:36.622205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170833bb-87f2-478b-b2f0-0c1224465289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:36.622480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170833bb-87f2-478b-b2f0-0c1224465289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:36.632597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170833bb-87f2-478b-b2f0-0c1224465289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:36.634043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170833bb-87f2-478b-b2f0-0c1224465289 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:36.645023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1079/2158] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:56:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:56:41.082707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9a1a-bde4-4479-94fc-ac386c6cdfbf None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:41.083272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9a1a-bde4-4479-94fc-ac386c6cdfbf None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:41.083398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9a1a-bde4-4479-94fc-ac386c6cdfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:41.104217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-f31b9a1a-bde4-4479-94fc-ac386c6cdfbf None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:41.105653 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f31b9a1a-bde4-4479-94fc-ac386c6cdfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:41.186779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1080/2159] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 05:56:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:41.199209 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170833bb-87f2-478b-b2f0-0c1224465289 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:41.200653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-170833bb-87f2-478b-b2f0-0c1224465289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:41.274048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170833bb-87f2-478b-b2f0-0c1224465289 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:41.279162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa6e0b-f79b-4006-aaab-a232d686709d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:41.280047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa6e0b-f79b-4006-aaab-a232d686709d service swift] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:41.280339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa6e0b-f79b-4006-aaab-a232d686709d service swift] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:41.280618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa6e0b-f79b-4006-aaab-a232d686709d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:41.290186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa6e0b-f79b-4006-aaab-a232d686709d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:41.291937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa6e0b-f79b-4006-aaab-a232d686709d service swift] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:41.301384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1080/2160] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 05:56:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 05:56:42.190403 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b9a1a-bde4-4479-94fc-ac386c6cdfbf None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:42.192117 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f31b9a1a-bde4-4479-94fc-ac386c6cdfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:42.262747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b9a1a-bde4-4479-94fc-ac386c6cdfbf None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:42.267825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:42.271395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:42.271395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:42.271395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] PATH_INFO: `/v3/users/d9fd428284ac4760a1c46fc2b68d1892` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:42.271395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9fd428284ac4760a1c46fc2b68d1892)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:42.273228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:42.369623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9fd428284ac4760a1c46fc2b68d1892'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:42.381608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9fd428284ac4760a1c46fc2b68d1892'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:42.408504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9fd428284ac4760a1c46fc2b68d1892'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:42.409254 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9fd428284ac4760a1c46fc2b68d1892'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:42.411721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9fd428284ac4760a1c46fc2b68d1892'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:42.437780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1081/2161] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:56:42 2022] DELETE /identity/v3/users/d9fd428284ac4760a1c46fc2b68d1892 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:56:42.443018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc80e4e-f8fa-4a44-b590-c8188aea0bb0 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:42.444168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc80e4e-f8fa-4a44-b590-c8188aea0bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:42.444653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc80e4e-f8fa-4a44-b590-c8188aea0bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:42.478231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-8dc80e4e-f8fa-4a44-b590-c8188aea0bb0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:42.479990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8dc80e4e-f8fa-4a44-b590-c8188aea0bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:42.558665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:42.604012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1081/2162] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:56:42.614909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:42.616551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:42.632523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:42.728260 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfa7f8-dfdf-44b7-9509-16d33a288587 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:42.732444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2d815-b950-4563-a3f1-9139b0143dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:42.733496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2d815-b950-4563-a3f1-9139b0143dec service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:42.733781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2d815-b950-4563-a3f1-9139b0143dec service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:42.734055 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2d815-b950-4563-a3f1-9139b0143dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:42.749190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2d815-b950-4563-a3f1-9139b0143dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:42.750786 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2d815-b950-4563-a3f1-9139b0143dec service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:42.775584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1082/2163] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:56:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:56:43.435811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc80e4e-f8fa-4a44-b590-c8188aea0bb0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:43.444349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8dc80e4e-f8fa-4a44-b590-c8188aea0bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:43.471400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc80e4e-f8fa-4a44-b590-c8188aea0bb0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:43.476397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:43.477632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:43.477975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:43.478413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] PATH_INFO: `/v3/projects/9b45aee732d74ce98573686d0c99b048` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:43.482084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b45aee732d74ce98573686d0c99b048)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:43.482084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:43.558541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b45aee732d74ce98573686d0c99b048', 'request_id': 'req-5b5c8794-af52-4b5c-bb48-98ece80ff47f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:43.564746 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b45aee732d74ce98573686d0c99b048', 'request_id': 'req-5b5c8794-af52-4b5c-bb48-98ece80ff47f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:43.570616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b45aee732d74ce98573686d0c99b048', 'request_id': 'req-5b5c8794-af52-4b5c-bb48-98ece80ff47f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:43.590484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] The token cache is being invalidate because project 9b45aee732d74ce98573686d0c99b048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:56:43.590834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:43.591709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-5b5c8794-af52-4b5c-bb48-98ece80ff47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:43.598671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1082/2164] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:56:43 2022] DELETE /identity/v3/projects/9b45aee732d74ce98573686d0c99b048 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:56:43.603526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16d808-a64c-4e88-a976-4be80b2e2bc9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:43.604764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16d808-a64c-4e88-a976-4be80b2e2bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:43.605227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16d808-a64c-4e88-a976-4be80b2e2bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:43.669604 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-9a16d808-a64c-4e88-a976-4be80b2e2bc9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:43.671308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-9a16d808-a64c-4e88-a976-4be80b2e2bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:43.747746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1083/2165] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:43.755594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f279a-e2a4-4883-bf5d-c3cb3b1be1ed None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:43.755966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f279a-e2a4-4883-bf5d-c3cb3b1be1ed None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:43.756341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f279a-e2a4-4883-bf5d-c3cb3b1be1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:43.779345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-4e9f279a-e2a4-4883-bf5d-c3cb3b1be1ed None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:43.781976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4e9f279a-e2a4-4883-bf5d-c3cb3b1be1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:43.791305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1083/2166] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:43.796144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16d808-a64c-4e88-a976-4be80b2e2bc9 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:43.806853 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16d808-a64c-4e88-a976-4be80b2e2bc9 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:43.812838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312469f9-60de-4c2a-850f-d9c0d7083472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:43.813584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312469f9-60de-4c2a-850f-d9c0d7083472 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:43.813873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312469f9-60de-4c2a-850f-d9c0d7083472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:43.814152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312469f9-60de-4c2a-850f-d9c0d7083472 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:43.815057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312469f9-60de-4c2a-850f-d9c0d7083472 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:56:43.815421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312469f9-60de-4c2a-850f-d9c0d7083472 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:43.817288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312469f9-60de-4c2a-850f-d9c0d7083472 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:43.830743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1084/2167] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:56:43.836443 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f279a-e2a4-4883-bf5d-c3cb3b1be1ed None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:43.847006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f279a-e2a4-4883-bf5d-c3cb3b1be1ed None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:43.853261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647e6d8-5b7b-4cd8-9276-1b33980421db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:43.853947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647e6d8-5b7b-4cd8-9276-1b33980421db admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:43.854290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647e6d8-5b7b-4cd8-9276-1b33980421db admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:43.854634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647e6d8-5b7b-4cd8-9276-1b33980421db admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:43.855203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647e6d8-5b7b-4cd8-9276-1b33980421db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:43.857163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647e6d8-5b7b-4cd8-9276-1b33980421db admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:43.891531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1084/2168] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:56:43 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:56:43.897415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312469f9-60de-4c2a-850f-d9c0d7083472 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:43.911195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312469f9-60de-4c2a-850f-d9c0d7083472 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:44.009227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957ca50-5123-44b0-af04-a4b912531ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:44.010242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957ca50-5123-44b0-af04-a4b912531ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:44.010555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957ca50-5123-44b0-af04-a4b912531ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:44.011133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957ca50-5123-44b0-af04-a4b912531ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:44.011878 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957ca50-5123-44b0-af04-a4b912531ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:44.013854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957ca50-5123-44b0-af04-a4b912531ee7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:44.038321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:44.048839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647e6d8-5b7b-4cd8-9276-1b33980421db admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:44.055405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1085/2169] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:56:43 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:56:44.072325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647e6d8-5b7b-4cd8-9276-1b33980421db admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:44.088030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6980a61-543e-449e-85d6-d7a3b91dd2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:44.088768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6980a61-543e-449e-85d6-d7a3b91dd2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:44.089033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6980a61-543e-449e-85d6-d7a3b91dd2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:44.089299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6980a61-543e-449e-85d6-d7a3b91dd2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:44.090420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6980a61-543e-449e-85d6-d7a3b91dd2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:44.092529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6980a61-543e-449e-85d6-d7a3b91dd2c4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:44.118280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1085/2170] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:56:44.126515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957ca50-5123-44b0-af04-a4b912531ee7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:44.140339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957ca50-5123-44b0-af04-a4b912531ee7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:44.157331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a218-c370-494a-b50e-71b2e05e7042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:44.158316 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a218-c370-494a-b50e-71b2e05e7042 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:44.158807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a218-c370-494a-b50e-71b2e05e7042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:44.159330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a218-c370-494a-b50e-71b2e05e7042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:44.160356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8a218-c370-494a-b50e-71b2e05e7042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:44.163089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8a218-c370-494a-b50e-71b2e05e7042 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:44.192748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1086/2171] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:56:44.197124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6980a61-543e-449e-85d6-d7a3b91dd2c4 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:44.215807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6980a61-543e-449e-85d6-d7a3b91dd2c4 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:44.224696 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7072e-b9cd-402f-b875-e64cbbc82783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:44.225972 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7072e-b9cd-402f-b875-e64cbbc82783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:44.226437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7072e-b9cd-402f-b875-e64cbbc82783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:44.226894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7072e-b9cd-402f-b875-e64cbbc82783 admin admin] PATH_INFO: `/v3/projects/870b7472a64540fb921bef25e3b7d813/users/e2ef854e949742828babe250698ffc48/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:44.256889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7072e-b9cd-402f-b875-e64cbbc82783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=870b7472a64540fb921bef25e3b7d813, user_id=e2ef854e949742828babe250698ffc48, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:44.259776 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7072e-b9cd-402f-b875-e64cbbc82783 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:44.295319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1086/2172] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:56:44 2022] PUT /identity/v3/projects/870b7472a64540fb921bef25e3b7d813/users/e2ef854e949742828babe250698ffc48/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:56:44.308001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107626f-406d-4c35-9235-f65db324b647 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:44.308722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107626f-406d-4c35-9235-f65db324b647 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:44.309242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107626f-406d-4c35-9235-f65db324b647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:44.379230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-b107626f-406d-4c35-9235-f65db324b647 None None] MFA Rules not processed for user `e2ef854e949742828babe250698ffc48`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:44.380803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b107626f-406d-4c35-9235-f65db324b647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:44.562807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1087/2173] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:56:44.578132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7072e-b9cd-402f-b875-e64cbbc82783 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:44.580077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-47c7072e-b9cd-402f-b875-e64cbbc82783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:44.724373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7072e-b9cd-402f-b875-e64cbbc82783 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:44.733256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2407a-69e9-4949-bd55-85d5ab659c59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:44.734499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2407a-69e9-4949-bd55-85d5ab659c59 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:44.734892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2407a-69e9-4949-bd55-85d5ab659c59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:44.735168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2407a-69e9-4949-bd55-85d5ab659c59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:44.749791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2407a-69e9-4949-bd55-85d5ab659c59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:44.751474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2407a-69e9-4949-bd55-85d5ab659c59 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:44.783066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1087/2174] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:56:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:56:54.554384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=37 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:54.759094 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:56:55.660075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107626f-406d-4c35-9235-f65db324b647 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:55.731467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107626f-406d-4c35-9235-f65db324b647 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:55.736111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:55.737096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:55.737385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:55.737746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] PATH_INFO: `/v3/users/e2ef854e949742828babe250698ffc48` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:55.740113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ef854e949742828babe250698ffc48)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:55.741077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:55.815469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ef854e949742828babe250698ffc48'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:55.826735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ef854e949742828babe250698ffc48'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:55.837381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ef854e949742828babe250698ffc48'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:55.838634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ef854e949742828babe250698ffc48'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:55.839607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ef854e949742828babe250698ffc48'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:55.869676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1088/2175] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:56:55 2022] DELETE /identity/v3/users/e2ef854e949742828babe250698ffc48 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:56:55.874489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66c0ff-b73e-40ca-bda4-d750f718b6f5 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:55.875021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66c0ff-b73e-40ca-bda4-d750f718b6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:55.875021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66c0ff-b73e-40ca-bda4-d750f718b6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:55.894501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6a66c0ff-b73e-40ca-bda4-d750f718b6f5 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:56:55.896490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6a66c0ff-b73e-40ca-bda4-d750f718b6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:55.963840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1088/2176] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:56:55.973649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:55.975106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:56.037273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c54a40-0d35-4873-b9ef-fd17343f4ec9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:56.042634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e495e75-3d17-486a-809e-9d344ac05b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:56.043641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e495e75-3d17-486a-809e-9d344ac05b9b service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:56.044110 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e495e75-3d17-486a-809e-9d344ac05b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:56.044547 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e495e75-3d17-486a-809e-9d344ac05b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:56.055748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e495e75-3d17-486a-809e-9d344ac05b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:56.057182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e495e75-3d17-486a-809e-9d344ac05b9b service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:56.066528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1089/2177] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:56:56.371822 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66c0ff-b73e-40ca-bda4-d750f718b6f5 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:56:56.373377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6a66c0ff-b73e-40ca-bda4-d750f718b6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:56:56.388287 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66c0ff-b73e-40ca-bda4-d750f718b6f5 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:56:56.393643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:56:56.394595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:56:56.394945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:56:56.395331 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] PATH_INFO: `/v3/projects/870b7472a64540fb921bef25e3b7d813` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:56:56.396814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=870b7472a64540fb921bef25e3b7d813)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:56:56.399060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:56:56.459553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '870b7472a64540fb921bef25e3b7d813', 'request_id': 'req-7c32d76c-14fc-47fb-b093-00f114aaa393'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:56.466686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '870b7472a64540fb921bef25e3b7d813', 'request_id': 'req-7c32d76c-14fc-47fb-b093-00f114aaa393'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:56.472890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '870b7472a64540fb921bef25e3b7d813', 'request_id': 'req-7c32d76c-14fc-47fb-b093-00f114aaa393'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:56.488159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] The token cache is being invalidate because project 870b7472a64540fb921bef25e3b7d813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:56:56.488450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:56.489067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-7c32d76c-14fc-47fb-b093-00f114aaa393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:56:56.491142 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1089/2178] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:56:56 2022] DELETE /identity/v3/projects/870b7472a64540fb921bef25e3b7d813 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:04.150619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e495e75-3d17-486a-809e-9d344ac05b9b service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:04.152123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8e495e75-3d17-486a-809e-9d344ac05b9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:04.179675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:04.252379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e495e75-3d17-486a-809e-9d344ac05b9b service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:04.320107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c525ee-3266-4acd-9846-c04f7e3b0f53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:04.320779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c525ee-3266-4acd-9846-c04f7e3b0f53 service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:04.321065 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c525ee-3266-4acd-9846-c04f7e3b0f53 service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:04.321348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c525ee-3266-4acd-9846-c04f7e3b0f53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:04.323157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-90c525ee-3266-4acd-9846-c04f7e3b0f53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:04.334496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c525ee-3266-4acd-9846-c04f7e3b0f53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:04.336135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c525ee-3266-4acd-9846-c04f7e3b0f53 service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:04.346570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1090/2179] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:57:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:57:06.485585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:12.589385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3b923-dc45-4b73-b2b9-5faf99f422c8 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:12.589988 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3b923-dc45-4b73-b2b9-5faf99f422c8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:12.589988 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3b923-dc45-4b73-b2b9-5faf99f422c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:12.617597 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-20c3b923-dc45-4b73-b2b9-5faf99f422c8 None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:12.618794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-20c3b923-dc45-4b73-b2b9-5faf99f422c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:12.701172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1090/2180] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 05:57:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:12.715831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c525ee-3266-4acd-9846-c04f7e3b0f53 service placement] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:12.717364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-90c525ee-3266-4acd-9846-c04f7e3b0f53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:12.790071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c525ee-3266-4acd-9846-c04f7e3b0f53 service placement] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:12.796685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130f6ce-854b-4cab-9e2e-7cbe06196185 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:12.797357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130f6ce-854b-4cab-9e2e-7cbe06196185 service swift] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:12.797642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130f6ce-854b-4cab-9e2e-7cbe06196185 service swift] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:12.797914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130f6ce-854b-4cab-9e2e-7cbe06196185 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:12.807120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130f6ce-854b-4cab-9e2e-7cbe06196185 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:12.808779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130f6ce-854b-4cab-9e2e-7cbe06196185 service swift] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:12.817732 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1091/2181] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 05:57:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 05:57:12.946593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c4037-548d-45ee-ba69-a16f0aa993dd None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:12.947084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c4037-548d-45ee-ba69-a16f0aa993dd None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:12.947401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c4037-548d-45ee-ba69-a16f0aa993dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:12.967806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d44c4037-548d-45ee-ba69-a16f0aa993dd None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:12.968930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d44c4037-548d-45ee-ba69-a16f0aa993dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:12.976578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1091/2182] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 4 05:57:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:12.993388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130f6ce-854b-4cab-9e2e-7cbe06196185 service swift] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:13.006094 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130f6ce-854b-4cab-9e2e-7cbe06196185 service swift] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:13.011590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeda316-6ba2-4034-8a13-57b1151a24bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:13.012291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeda316-6ba2-4034-8a13-57b1151a24bd service swift] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:13.012575 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeda316-6ba2-4034-8a13-57b1151a24bd service swift] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:13.012850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeda316-6ba2-4034-8a13-57b1151a24bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:13.026203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeda316-6ba2-4034-8a13-57b1151a24bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:13.028050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeda316-6ba2-4034-8a13-57b1151a24bd service swift] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:13.037849 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1092/2183] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 05:57:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 05:57:14.868032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b03e1c-61fb-408f-9b63-b2c5ee0b489d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:14.868607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b03e1c-61fb-408f-9b63-b2c5ee0b489d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:14.868607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b03e1c-61fb-408f-9b63-b2c5ee0b489d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:14.890632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-69b03e1c-61fb-408f-9b63-b2c5ee0b489d None None] MFA Rules not processed for user `93ae888c6e364e2e96ff52e63415a0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:14.892221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-69b03e1c-61fb-408f-9b63-b2c5ee0b489d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:14.953382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1092/2184] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:23.033832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:24.945847 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=60 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:32.406972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc5f3d-69bf-4568-9f8e-f4d1be6e9255 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:32.407480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc5f3d-69bf-4568-9f8e-f4d1be6e9255 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:32.407595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc5f3d-69bf-4568-9f8e-f4d1be6e9255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:32.459068 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-53bc5f3d-69bf-4568-9f8e-f4d1be6e9255 None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:32.460330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-53bc5f3d-69bf-4568-9f8e-f4d1be6e9255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:32.479204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1093/2185] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 4 05:57:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:32.492283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b03e1c-61fb-408f-9b63-b2c5ee0b489d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:32.516188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b03e1c-61fb-408f-9b63-b2c5ee0b489d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:32.522091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b8b29-3af6-41a2-988b-f2a01a0429c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:32.522753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b8b29-3af6-41a2-988b-f2a01a0429c7 service swift] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:32.523056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b8b29-3af6-41a2-988b-f2a01a0429c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:32.523396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b8b29-3af6-41a2-988b-f2a01a0429c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:32.532145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b8b29-3af6-41a2-988b-f2a01a0429c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:32.533753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b8b29-3af6-41a2-988b-f2a01a0429c7 service swift] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:32.542010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1093/2186] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 05:57:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 05:57:34.494943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc5f3d-69bf-4568-9f8e-f4d1be6e9255 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:34.496501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-53bc5f3d-69bf-4568-9f8e-f4d1be6e9255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:34.566900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc5f3d-69bf-4568-9f8e-f4d1be6e9255 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:34.571854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:34.572744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:34.573029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:34.573298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] PATH_INFO: `/v3/users/9dd9a1603e2340eba37ae8699f40b9f6` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:34.574583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dd9a1603e2340eba37ae8699f40b9f6)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:34.576652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:34.627988 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dd9a1603e2340eba37ae8699f40b9f6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:34.634774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dd9a1603e2340eba37ae8699f40b9f6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:34.643650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dd9a1603e2340eba37ae8699f40b9f6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:34.644266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dd9a1603e2340eba37ae8699f40b9f6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:34.644866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f5362f1b-2b44-4260-b8de-e30c698078a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dd9a1603e2340eba37ae8699f40b9f6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:34.661748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1094/2187] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:57:34 2022] DELETE /identity/v3/users/9dd9a1603e2340eba37ae8699f40b9f6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:35.093707 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b8b29-3af6-41a2-988b-f2a01a0429c7 service swift] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:35.107946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b8b29-3af6-41a2-988b-f2a01a0429c7 service swift] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:35.167729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:35.168915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.169214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.169492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] PATH_INFO: `/v3/projects/ee9a610826bc47b395a26f07ad375965` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.170693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9a610826bc47b395a26f07ad375965)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:35.172757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:35.220169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9a610826bc47b395a26f07ad375965', 'request_id': 'req-651dfd5c-ee50-4497-8f57-07cc191b5866'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:35.226224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9a610826bc47b395a26f07ad375965', 'request_id': 'req-651dfd5c-ee50-4497-8f57-07cc191b5866'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:35.232434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9a610826bc47b395a26f07ad375965', 'request_id': 'req-651dfd5c-ee50-4497-8f57-07cc191b5866'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:35.247102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] The token cache is being invalidate because project ee9a610826bc47b395a26f07ad375965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:57:35.247451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:35.248075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-651dfd5c-ee50-4497-8f57-07cc191b5866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:35.249643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1094/2188] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:57:35 2022] DELETE /identity/v3/projects/ee9a610826bc47b395a26f07ad375965 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:35.255916 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6718e-8d4c-4e00-b7d0-8197a47ca1bc None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.256205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6718e-8d4c-4e00-b7d0-8197a47ca1bc None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.256480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6718e-8d4c-4e00-b7d0-8197a47ca1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.275219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-f5b6718e-8d4c-4e00-b7d0-8197a47ca1bc None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:35.276395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f5b6718e-8d4c-4e00-b7d0-8197a47ca1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:35.313108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:35.345594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1095/2189] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:57:35.353080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e1079-e76e-451a-a078-79e4d083511f None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.353386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e1079-e76e-451a-a078-79e4d083511f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.353663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e1079-e76e-451a-a078-79e4d083511f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.372491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-212e1079-e76e-451a-a078-79e4d083511f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:35.373708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-212e1079-e76e-451a-a078-79e4d083511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:35.382447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1095/2190] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:35.386933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6718e-8d4c-4e00-b7d0-8197a47ca1bc None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:35.398636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6718e-8d4c-4e00-b7d0-8197a47ca1bc None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:35.403855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b595c1b-e267-451e-870e-05b469145119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:35.404719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595c1b-e267-451e-870e-05b469145119 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.405045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595c1b-e267-451e-870e-05b469145119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.405316 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595c1b-e267-451e-870e-05b469145119 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.406045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b595c1b-e267-451e-870e-05b469145119 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:57:35.406346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b595c1b-e267-451e-870e-05b469145119 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:35.407973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b595c1b-e267-451e-870e-05b469145119 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:35.420195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1096/2191] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:57:35.424638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e1079-e76e-451a-a078-79e4d083511f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:35.434497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e1079-e76e-451a-a078-79e4d083511f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:35.440392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23cd8-ad0e-4dcd-afb0-258daae1cbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:35.441048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23cd8-ad0e-4dcd-afb0-258daae1cbe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.441334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23cd8-ad0e-4dcd-afb0-258daae1cbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.441608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23cd8-ad0e-4dcd-afb0-258daae1cbe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.442303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae23cd8-ad0e-4dcd-afb0-258daae1cbe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:35.444295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae23cd8-ad0e-4dcd-afb0-258daae1cbe8 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:35.477536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1096/2192] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:57:35 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:57:35.479308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b595c1b-e267-451e-870e-05b469145119 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:35.495273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b595c1b-e267-451e-870e-05b469145119 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:35.555227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3371b3-c0ea-4315-a441-5dfed6c9e164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:35.555867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3371b3-c0ea-4315-a441-5dfed6c9e164 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.556152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3371b3-c0ea-4315-a441-5dfed6c9e164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.556427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3371b3-c0ea-4315-a441-5dfed6c9e164 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.557204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3371b3-c0ea-4315-a441-5dfed6c9e164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:35.559170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3371b3-c0ea-4315-a441-5dfed6c9e164 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:35.580955 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1097/2193] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:57:35 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:57:35.585211 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23cd8-ad0e-4dcd-afb0-258daae1cbe8 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:35.596167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23cd8-ad0e-4dcd-afb0-258daae1cbe8 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:35.601491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d679077-f4de-4964-868c-d01da3f44ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:35.602162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d679077-f4de-4964-868c-d01da3f44ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.602447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d679077-f4de-4964-868c-d01da3f44ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.602723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d679077-f4de-4964-868c-d01da3f44ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.603515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d679077-f4de-4964-868c-d01da3f44ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:35.605156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d679077-f4de-4964-868c-d01da3f44ebf admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:35.615881 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1097/2194] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:57:35.619840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3371b3-c0ea-4315-a441-5dfed6c9e164 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:35.631703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3371b3-c0ea-4315-a441-5dfed6c9e164 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:35.640155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93f6e1-e296-467c-98ac-49c991d09b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:35.640889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93f6e1-e296-467c-98ac-49c991d09b28 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.641259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93f6e1-e296-467c-98ac-49c991d09b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.641633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93f6e1-e296-467c-98ac-49c991d09b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.642373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93f6e1-e296-467c-98ac-49c991d09b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:35.644665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93f6e1-e296-467c-98ac-49c991d09b28 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:35.659406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1098/2195] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:57:35.663632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d679077-f4de-4964-868c-d01da3f44ebf admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:35.674768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d679077-f4de-4964-868c-d01da3f44ebf admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:35.680729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ec44d-03f9-48bc-80da-a6e1d09510e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:35.681617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ec44d-03f9-48bc-80da-a6e1d09510e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.681890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ec44d-03f9-48bc-80da-a6e1d09510e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.682171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ec44d-03f9-48bc-80da-a6e1d09510e1 admin admin] PATH_INFO: `/v3/projects/f39270da4b5342bb85f17d8dd64b5365/users/4fcda98b8c4a415d9cd92a5889b73963/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.697398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ec44d-03f9-48bc-80da-a6e1d09510e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f39270da4b5342bb85f17d8dd64b5365, user_id=4fcda98b8c4a415d9cd92a5889b73963, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:35.700157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ec44d-03f9-48bc-80da-a6e1d09510e1 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:35.715718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1098/2196] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:57:35 2022] PUT /identity/v3/projects/f39270da4b5342bb85f17d8dd64b5365/users/4fcda98b8c4a415d9cd92a5889b73963/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:35.726420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2fa4-a7ec-4eaa-b64f-711af02fbedb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.726724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2fa4-a7ec-4eaa-b64f-711af02fbedb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.726999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2fa4-a7ec-4eaa-b64f-711af02fbedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.754622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-547e2fa4-a7ec-4eaa-b64f-711af02fbedb None None] MFA Rules not processed for user `4fcda98b8c4a415d9cd92a5889b73963`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:35.755846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-547e2fa4-a7ec-4eaa-b64f-711af02fbedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:35.825518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1099/2197] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:57:35.837193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ec44d-03f9-48bc-80da-a6e1d09510e1 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:35.839424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-980ec44d-03f9-48bc-80da-a6e1d09510e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:35.844883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:35.922969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ec44d-03f9-48bc-80da-a6e1d09510e1 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:35.927309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc787299-0ae7-4d81-b84f-bdcf11e668a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:35.927980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc787299-0ae7-4d81-b84f-bdcf11e668a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:35.928261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc787299-0ae7-4d81-b84f-bdcf11e668a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:35.928563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc787299-0ae7-4d81-b84f-bdcf11e668a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:35.937692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc787299-0ae7-4d81-b84f-bdcf11e668a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:35.939143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc787299-0ae7-4d81-b84f-bdcf11e668a6 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:35.948347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1099/2198] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:57:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:57:41.676877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e2fa4-a7ec-4eaa-b64f-711af02fbedb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:41.720391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:41.746131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e2fa4-a7ec-4eaa-b64f-711af02fbedb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:41.750663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:41.751594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:41.751893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:41.752161 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] PATH_INFO: `/v3/users/4fcda98b8c4a415d9cd92a5889b73963` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:41.753344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fcda98b8c4a415d9cd92a5889b73963)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:41.755317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:41.802963 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fcda98b8c4a415d9cd92a5889b73963'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:41.809973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fcda98b8c4a415d9cd92a5889b73963'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:41.819696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fcda98b8c4a415d9cd92a5889b73963'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:41.820354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fcda98b8c4a415d9cd92a5889b73963'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:41.821165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fcda98b8c4a415d9cd92a5889b73963'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:41.840763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1100/2199] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:57:41 2022] DELETE /identity/v3/users/4fcda98b8c4a415d9cd92a5889b73963 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:41.846042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01f6cc-b425-4cb2-98ad-6d9bf4030166 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:41.846540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01f6cc-b425-4cb2-98ad-6d9bf4030166 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:41.846619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01f6cc-b425-4cb2-98ad-6d9bf4030166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:41.866269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-7c01f6cc-b425-4cb2-98ad-6d9bf4030166 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:41.867532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7c01f6cc-b425-4cb2-98ad-6d9bf4030166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:41.934701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1100/2200] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:41.946594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:41.948036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:42.015347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0ed63-e8d0-491b-a2a8-23ebd65722f2 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:42.018939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797c5ce-4119-403f-8412-3e1e5c32ae1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:42.019637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797c5ce-4119-403f-8412-3e1e5c32ae1d service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:42.019917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797c5ce-4119-403f-8412-3e1e5c32ae1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:42.020185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797c5ce-4119-403f-8412-3e1e5c32ae1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:42.030028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797c5ce-4119-403f-8412-3e1e5c32ae1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:42.031562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797c5ce-4119-403f-8412-3e1e5c32ae1d service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:42.041415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1101/2201] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:57:42.324356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01f6cc-b425-4cb2-98ad-6d9bf4030166 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:42.325937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7c01f6cc-b425-4cb2-98ad-6d9bf4030166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:42.348910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01f6cc-b425-4cb2-98ad-6d9bf4030166 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:42.354322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:42.355230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:42.355560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:42.355832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] PATH_INFO: `/v3/projects/f39270da4b5342bb85f17d8dd64b5365` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:42.357031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f39270da4b5342bb85f17d8dd64b5365)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:42.360242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:42.409409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f39270da4b5342bb85f17d8dd64b5365', 'request_id': 'req-3dee495b-f745-4c4f-bf37-60f918233fa8'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:42.416501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f39270da4b5342bb85f17d8dd64b5365', 'request_id': 'req-3dee495b-f745-4c4f-bf37-60f918233fa8'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:42.423043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f39270da4b5342bb85f17d8dd64b5365', 'request_id': 'req-3dee495b-f745-4c4f-bf37-60f918233fa8'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:42.442800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] The token cache is being invalidate because project f39270da4b5342bb85f17d8dd64b5365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:57:42.443091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:42.443933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3dee495b-f745-4c4f-bf37-60f918233fa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:42.445681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1101/2202] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:57:42 2022] DELETE /identity/v3/projects/f39270da4b5342bb85f17d8dd64b5365 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:42.450987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be715eeb-0f45-4c01-aaea-247918aeee13 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:42.451307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be715eeb-0f45-4c01-aaea-247918aeee13 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:42.451589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be715eeb-0f45-4c01-aaea-247918aeee13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:42.476171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-be715eeb-0f45-4c01-aaea-247918aeee13 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:42.480072 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-be715eeb-0f45-4c01-aaea-247918aeee13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:42.559705 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:42.572245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1102/2203] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:57:42.579835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017f992-8c52-4d52-8de0-5caf89ea1c88 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:42.580127 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017f992-8c52-4d52-8de0-5caf89ea1c88 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:42.580407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017f992-8c52-4d52-8de0-5caf89ea1c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:42.600413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-5017f992-8c52-4d52-8de0-5caf89ea1c88 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:42.601642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5017f992-8c52-4d52-8de0-5caf89ea1c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:42.611295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1102/2204] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:42.614745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be715eeb-0f45-4c01-aaea-247918aeee13 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:42.625668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be715eeb-0f45-4c01-aaea-247918aeee13 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:42.631848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a99ed-ba6b-4d1d-8d0c-e72fcf8dec54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:42.632759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a99ed-ba6b-4d1d-8d0c-e72fcf8dec54 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:42.633218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a99ed-ba6b-4d1d-8d0c-e72fcf8dec54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:42.633642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a99ed-ba6b-4d1d-8d0c-e72fcf8dec54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:42.635200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a99ed-ba6b-4d1d-8d0c-e72fcf8dec54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:57:42.635691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a99ed-ba6b-4d1d-8d0c-e72fcf8dec54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:42.637515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a99ed-ba6b-4d1d-8d0c-e72fcf8dec54 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:42.652596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1103/2205] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:57:42.657704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017f992-8c52-4d52-8de0-5caf89ea1c88 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:42.664180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:42.670434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017f992-8c52-4d52-8de0-5caf89ea1c88 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:42.678156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e9b81-c08b-4412-8932-52295c61100d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:42.679270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e9b81-c08b-4412-8932-52295c61100d admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:42.679660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e9b81-c08b-4412-8932-52295c61100d admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:42.680014 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e9b81-c08b-4412-8932-52295c61100d admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:42.681657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e9b81-c08b-4412-8932-52295c61100d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:42.684246 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e9b81-c08b-4412-8932-52295c61100d admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:42.723130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1103/2206] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:57:42 2022] POST /identity/v3/projects => generated 399 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:57:42.724508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a99ed-ba6b-4d1d-8d0c-e72fcf8dec54 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:42.738366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a99ed-ba6b-4d1d-8d0c-e72fcf8dec54 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:42.817760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820345e-4c53-439f-aff9-b9064bacceb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:42.819760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820345e-4c53-439f-aff9-b9064bacceb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:42.820115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820345e-4c53-439f-aff9-b9064bacceb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:42.820469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820345e-4c53-439f-aff9-b9064bacceb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:42.821269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820345e-4c53-439f-aff9-b9064bacceb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:42.823319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820345e-4c53-439f-aff9-b9064bacceb7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:42.849643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1104/2207] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:57:42 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:57:42.853781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e9b81-c08b-4412-8932-52295c61100d admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:42.867591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e9b81-c08b-4412-8932-52295c61100d admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:42.871348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cb174-64aa-4d76-b0f5-8c03a08538ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:42.872185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cb174-64aa-4d76-b0f5-8c03a08538ab admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:42.872620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cb174-64aa-4d76-b0f5-8c03a08538ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:42.872966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cb174-64aa-4d76-b0f5-8c03a08538ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:42.873838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cb174-64aa-4d76-b0f5-8c03a08538ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:42.875585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cb174-64aa-4d76-b0f5-8c03a08538ab admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:42.886844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1104/2208] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 05:57:42.890866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820345e-4c53-439f-aff9-b9064bacceb7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:42.901391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820345e-4c53-439f-aff9-b9064bacceb7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:42.906142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207327c-ed04-47b3-96c3-95353cd2635e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:42.906826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207327c-ed04-47b3-96c3-95353cd2635e admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:42.907112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207327c-ed04-47b3-96c3-95353cd2635e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:42.907425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207327c-ed04-47b3-96c3-95353cd2635e admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:42.908177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207327c-ed04-47b3-96c3-95353cd2635e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:42.909822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207327c-ed04-47b3-96c3-95353cd2635e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:42.921374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1105/2209] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:57:42.927184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cb174-64aa-4d76-b0f5-8c03a08538ab admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:42.945767 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cb174-64aa-4d76-b0f5-8c03a08538ab admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:42.950995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a0051-8f80-4843-9dfe-98d88dfda06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:42.951933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a0051-8f80-4843-9dfe-98d88dfda06f admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:42.952214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a0051-8f80-4843-9dfe-98d88dfda06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:42.952539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a0051-8f80-4843-9dfe-98d88dfda06f admin admin] PATH_INFO: `/v3/projects/f44975e470f54e59ac46e4d38620ee25/users/9348be8b82ef43ae985a8c50c167e4e4/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:42.969046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a0051-8f80-4843-9dfe-98d88dfda06f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44975e470f54e59ac46e4d38620ee25, user_id=9348be8b82ef43ae985a8c50c167e4e4, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:42.971801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a0051-8f80-4843-9dfe-98d88dfda06f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:42.991431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1105/2210] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:57:42 2022] PUT /identity/v3/projects/f44975e470f54e59ac46e4d38620ee25/users/9348be8b82ef43ae985a8c50c167e4e4/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:43.002383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e42d4a-6244-4975-af79-4ebabb09866d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:43.002938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e42d4a-6244-4975-af79-4ebabb09866d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:43.003442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e42d4a-6244-4975-af79-4ebabb09866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:43.039624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-b7e42d4a-6244-4975-af79-4ebabb09866d None None] MFA Rules not processed for user `9348be8b82ef43ae985a8c50c167e4e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:43.041025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b7e42d4a-6244-4975-af79-4ebabb09866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:43.131937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1106/2211] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:43.139527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a0051-8f80-4843-9dfe-98d88dfda06f admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:43.141306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-535a0051-8f80-4843-9dfe-98d88dfda06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:43.241323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a0051-8f80-4843-9dfe-98d88dfda06f admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:43.248488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47c3e5-c0c9-4258-9034-84d9b351b1fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:43.249529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47c3e5-c0c9-4258-9034-84d9b351b1fe service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:43.249917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47c3e5-c0c9-4258-9034-84d9b351b1fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:43.250262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47c3e5-c0c9-4258-9034-84d9b351b1fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:43.263476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47c3e5-c0c9-4258-9034-84d9b351b1fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:43.265160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47c3e5-c0c9-4258-9034-84d9b351b1fe service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:43.281600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1106/2212] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:57:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:57:51.524423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e42d4a-6244-4975-af79-4ebabb09866d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:51.581344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:51.593598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e42d4a-6244-4975-af79-4ebabb09866d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:51.598563 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:51.599507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:51.599786 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:51.600066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] PATH_INFO: `/v3/users/9348be8b82ef43ae985a8c50c167e4e4` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:51.601275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9348be8b82ef43ae985a8c50c167e4e4)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:51.603204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:51.656821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9348be8b82ef43ae985a8c50c167e4e4'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:51.663805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9348be8b82ef43ae985a8c50c167e4e4'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:51.673841 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9348be8b82ef43ae985a8c50c167e4e4'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:51.674577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9348be8b82ef43ae985a8c50c167e4e4'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:51.675218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9348be8b82ef43ae985a8c50c167e4e4'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:51.694842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1107/2213] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 05:57:51 2022] DELETE /identity/v3/users/9348be8b82ef43ae985a8c50c167e4e4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:51.700061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6e1e3-6ce3-45af-ae91-697245d90a13 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:51.700534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6e1e3-6ce3-45af-ae91-697245d90a13 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:51.700972 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6e1e3-6ce3-45af-ae91-697245d90a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:51.719844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-cfa6e1e3-6ce3-45af-ae91-697245d90a13 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:51.721139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cfa6e1e3-6ce3-45af-ae91-697245d90a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:51.785335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1107/2214] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:51.795443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:51.797017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:51.857354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e0010-738e-494f-8bbf-e95ce652c80a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:51.861134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde8787-a825-498b-ad2a-77340363807b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:51.861792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde8787-a825-498b-ad2a-77340363807b service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:51.862074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde8787-a825-498b-ad2a-77340363807b service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:51.862345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde8787-a825-498b-ad2a-77340363807b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:51.871518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde8787-a825-498b-ad2a-77340363807b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:51.872998 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde8787-a825-498b-ad2a-77340363807b service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:51.882366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1108/2215] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:57:52.162691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6e1e3-6ce3-45af-ae91-697245d90a13 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:52.164375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cfa6e1e3-6ce3-45af-ae91-697245d90a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:52.176988 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6e1e3-6ce3-45af-ae91-697245d90a13 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:52.181800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:52.182680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.182960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.183232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] PATH_INFO: `/v3/projects/f44975e470f54e59ac46e4d38620ee25` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.184518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f44975e470f54e59ac46e4d38620ee25)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:52.186449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:52.233269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:52.236447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44975e470f54e59ac46e4d38620ee25', 'request_id': 'req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:52.243812 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44975e470f54e59ac46e4d38620ee25', 'request_id': 'req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:52.249336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44975e470f54e59ac46e4d38620ee25', 'request_id': 'req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:52.262748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] The token cache is being invalidate because project f44975e470f54e59ac46e4d38620ee25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 05:57:52.263035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:52.263691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-1036b2c7-04b9-4a70-8c01-b9a69aafcaf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 05:57:52.265386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1108/2216] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 05:57:52 2022] DELETE /identity/v3/projects/f44975e470f54e59ac46e4d38620ee25 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:52.270640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680cc63-a178-40c2-a8b1-27cfdd74e17e None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.270925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680cc63-a178-40c2-a8b1-27cfdd74e17e None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.271196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680cc63-a178-40c2-a8b1-27cfdd74e17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.290216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-5680cc63-a178-40c2-a8b1-27cfdd74e17e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:52.291475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5680cc63-a178-40c2-a8b1-27cfdd74e17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:52.351857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1109/2217] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:52.360017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d2efc-82ab-423b-84e6-760e28117f3e None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.360272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d2efc-82ab-423b-84e6-760e28117f3e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.360545 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d2efc-82ab-423b-84e6-760e28117f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.378873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-076d2efc-82ab-423b-84e6-760e28117f3e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:52.380118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-076d2efc-82ab-423b-84e6-760e28117f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:52.388036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1109/2218] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:57:52.392565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680cc63-a178-40c2-a8b1-27cfdd74e17e None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:52.402155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680cc63-a178-40c2-a8b1-27cfdd74e17e None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:52.406829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0692f2-0a72-4fa2-a788-44cefc778568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:52.407534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0692f2-0a72-4fa2-a788-44cefc778568 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.407817 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0692f2-0a72-4fa2-a788-44cefc778568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.408092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0692f2-0a72-4fa2-a788-44cefc778568 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.408822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0692f2-0a72-4fa2-a788-44cefc778568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 05:57:52.409129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0692f2-0a72-4fa2-a788-44cefc778568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:52.410746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0692f2-0a72-4fa2-a788-44cefc778568 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:52.424070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1110/2219] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 05:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 05:57:52.428403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d2efc-82ab-423b-84e6-760e28117f3e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:52.438064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d2efc-82ab-423b-84e6-760e28117f3e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:52.443302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262f741-ba60-45a0-8f9a-aceb1f4b5c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:52.443974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f741-ba60-45a0-8f9a-aceb1f4b5c65 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.444255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f741-ba60-45a0-8f9a-aceb1f4b5c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.444527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f741-ba60-45a0-8f9a-aceb1f4b5c65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.445221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262f741-ba60-45a0-8f9a-aceb1f4b5c65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:52.447156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262f741-ba60-45a0-8f9a-aceb1f4b5c65 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:52.474662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1110/2220] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:57:52 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:57:52.478899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0692f2-0a72-4fa2-a788-44cefc778568 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:52.484187 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:52.492229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0692f2-0a72-4fa2-a788-44cefc778568 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:52.557417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada402-b961-48f8-accd-7708c0636ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:52.558058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada402-b961-48f8-accd-7708c0636ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.558374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada402-b961-48f8-accd-7708c0636ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.558636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada402-b961-48f8-accd-7708c0636ed8 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.559361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada402-b961-48f8-accd-7708c0636ed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:52.561412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada402-b961-48f8-accd-7708c0636ed8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:52.586699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1111/2221] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:57:52 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 05:57:52.588097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262f741-ba60-45a0-8f9a-aceb1f4b5c65 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:52.598581 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262f741-ba60-45a0-8f9a-aceb1f4b5c65 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:52.603943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1f3bc-8f30-40b3-bffd-7b973cda18e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:52.604611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1f3bc-8f30-40b3-bffd-7b973cda18e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.604895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1f3bc-8f30-40b3-bffd-7b973cda18e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.605170 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1f3bc-8f30-40b3-bffd-7b973cda18e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.605905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1f3bc-8f30-40b3-bffd-7b973cda18e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:52.607563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1f3bc-8f30-40b3-bffd-7b973cda18e7 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:52.618681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1111/2222] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:57:52.622987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada402-b961-48f8-accd-7708c0636ed8 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:52.633105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada402-b961-48f8-accd-7708c0636ed8 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:52.638161 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641200db-abec-4f90-b722-d04ee90ae61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:52.638869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641200db-abec-4f90-b722-d04ee90ae61c admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.639155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641200db-abec-4f90-b722-d04ee90ae61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.639456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641200db-abec-4f90-b722-d04ee90ae61c admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.640234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641200db-abec-4f90-b722-d04ee90ae61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:52.641816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641200db-abec-4f90-b722-d04ee90ae61c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:52.651903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1112/2223] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:57:52.656016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1f3bc-8f30-40b3-bffd-7b973cda18e7 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:52.666378 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1f3bc-8f30-40b3-bffd-7b973cda18e7 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:52.671720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4cf83-5f40-41b8-96d5-6b8503cbac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:52.672590 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4cf83-5f40-41b8-96d5-6b8503cbac24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.672871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4cf83-5f40-41b8-96d5-6b8503cbac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.673145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4cf83-5f40-41b8-96d5-6b8503cbac24 admin admin] PATH_INFO: `/v3/projects/8d701a66ecbb480fa08e97f304acf57b/users/6ac9756c145040e58f9d914ba5bcdfaa/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.687906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4cf83-5f40-41b8-96d5-6b8503cbac24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d701a66ecbb480fa08e97f304acf57b, user_id=6ac9756c145040e58f9d914ba5bcdfaa, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:52.690583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4cf83-5f40-41b8-96d5-6b8503cbac24 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:52.706175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1112/2224] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:57:52 2022] PUT /identity/v3/projects/8d701a66ecbb480fa08e97f304acf57b/users/6ac9756c145040e58f9d914ba5bcdfaa/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:52.711318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771727-9bf4-4df5-af48-402cb3b2bc88 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.711615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771727-9bf4-4df5-af48-402cb3b2bc88 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.711898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771727-9bf4-4df5-af48-402cb3b2bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.730490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-dd771727-9bf4-4df5-af48-402cb3b2bc88 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:52.731763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-dd771727-9bf4-4df5-af48-402cb3b2bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:52.795281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1113/2225] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:57:52.807891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4cf83-5f40-41b8-96d5-6b8503cbac24 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:52.809698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-32d4cf83-5f40-41b8-96d5-6b8503cbac24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:52.870623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4cf83-5f40-41b8-96d5-6b8503cbac24 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:52.874615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2e75a-3bed-478c-9985-15f4a5fe04b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:52.875411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2e75a-3bed-478c-9985-15f4a5fe04b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:52.875694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2e75a-3bed-478c-9985-15f4a5fe04b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:52.875969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2e75a-3bed-478c-9985-15f4a5fe04b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:52.885726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2e75a-3bed-478c-9985-15f4a5fe04b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:52.887272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2e75a-3bed-478c-9985-15f4a5fe04b6 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:52.897358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1113/2226] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:57:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:57:58.669911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e96f95-871a-40fe-91ef-053def4b1fa5 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:58.670455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e96f95-871a-40fe-91ef-053def4b1fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:58.670455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e96f95-871a-40fe-91ef-053def4b1fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:58.705671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-f4e96f95-871a-40fe-91ef-053def4b1fa5 None None] MFA Rules not processed for user `6ac9756c145040e58f9d914ba5bcdfaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:58.707444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f4e96f95-871a-40fe-91ef-053def4b1fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:58.806360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1114/2227] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:57:58.811323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2e75a-3bed-478c-9985-15f4a5fe04b6 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:58.821794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2e75a-3bed-478c-9985-15f4a5fe04b6 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:58.826531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af145d66-e355-4a62-8895-ebc0fe75b961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:58.827195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af145d66-e355-4a62-8895-ebc0fe75b961 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:58.827582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af145d66-e355-4a62-8895-ebc0fe75b961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:58.827859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af145d66-e355-4a62-8895-ebc0fe75b961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:58.828572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af145d66-e355-4a62-8895-ebc0fe75b961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:58.830529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af145d66-e355-4a62-8895-ebc0fe75b961 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:58.863131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1114/2228] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 05:57:58 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:57:58.864799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e96f95-871a-40fe-91ef-053def4b1fa5 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:58.876307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e96f95-871a-40fe-91ef-053def4b1fa5 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:58.885399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:58.937941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4565f7-42b9-4314-893a-052cce72fc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:58.938631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4565f7-42b9-4314-893a-052cce72fc80 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:58.938914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4565f7-42b9-4314-893a-052cce72fc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:58.939220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4565f7-42b9-4314-893a-052cce72fc80 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:58.939951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4565f7-42b9-4314-893a-052cce72fc80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:58.941891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4565f7-42b9-4314-893a-052cce72fc80 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:58.963900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1115/2229] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 05:57:58 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 05:57:58.968032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af145d66-e355-4a62-8895-ebc0fe75b961 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:58.978609 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af145d66-e355-4a62-8895-ebc0fe75b961 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:58.984182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b3fa7-5c4c-49c4-9afe-5ed354276909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:58.984894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b3fa7-5c4c-49c4-9afe-5ed354276909 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:58.985181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b3fa7-5c4c-49c4-9afe-5ed354276909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:58.985460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b3fa7-5c4c-49c4-9afe-5ed354276909 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:58.986197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b3fa7-5c4c-49c4-9afe-5ed354276909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:58.987878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b3fa7-5c4c-49c4-9afe-5ed354276909 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:58.999503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1115/2230] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:57:59.003950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4565f7-42b9-4314-893a-052cce72fc80 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:59.014283 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4565f7-42b9-4314-893a-052cce72fc80 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:59.018984 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e253bb-e31c-498a-847a-52e238082c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:59.019907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e253bb-e31c-498a-847a-52e238082c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:59.020187 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e253bb-e31c-498a-847a-52e238082c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:59.020458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e253bb-e31c-498a-847a-52e238082c0e admin admin] PATH_INFO: `/v3/projects/a3df8b054d4d4de5baa7778467f6e5a0/users/b1310c60f7624dfa83023cca199d5d30/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:59.036226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e253bb-e31c-498a-847a-52e238082c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3df8b054d4d4de5baa7778467f6e5a0, user_id=b1310c60f7624dfa83023cca199d5d30, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:59.039018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e253bb-e31c-498a-847a-52e238082c0e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:59.055206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1116/2231] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:57:59 2022] PUT /identity/v3/projects/a3df8b054d4d4de5baa7778467f6e5a0/users/b1310c60f7624dfa83023cca199d5d30/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:59.059763 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b3fa7-5c4c-49c4-9afe-5ed354276909 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:59.072572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b3fa7-5c4c-49c4-9afe-5ed354276909 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:59.115538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:57:59.138732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ad83e-5bf7-459c-8e9f-0ab6a00267c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:59.139453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ad83e-5bf7-459c-8e9f-0ab6a00267c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:59.139740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ad83e-5bf7-459c-8e9f-0ab6a00267c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:59.140117 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ad83e-5bf7-459c-8e9f-0ab6a00267c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:59.140903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ad83e-5bf7-459c-8e9f-0ab6a00267c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:59.142537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ad83e-5bf7-459c-8e9f-0ab6a00267c6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:59.152919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1116/2232] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 05:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 05:57:59.157556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e253bb-e31c-498a-847a-52e238082c0e admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:57:59.172778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e253bb-e31c-498a-847a-52e238082c0e admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:57:59.179164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63878b06-0f5a-4e2f-97bf-adcbd5f2892c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:57:59.180324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63878b06-0f5a-4e2f-97bf-adcbd5f2892c admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:59.180612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63878b06-0f5a-4e2f-97bf-adcbd5f2892c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:59.180892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63878b06-0f5a-4e2f-97bf-adcbd5f2892c admin admin] PATH_INFO: `/v3/projects/a3df8b054d4d4de5baa7778467f6e5a0/users/b1310c60f7624dfa83023cca199d5d30/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:59.183089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63878b06-0f5a-4e2f-97bf-adcbd5f2892c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3df8b054d4d4de5baa7778467f6e5a0, user_id=b1310c60f7624dfa83023cca199d5d30, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:57:59.186290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63878b06-0f5a-4e2f-97bf-adcbd5f2892c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:57:59.193456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808afb3-f398-4c9a-b622-e177ac644a47 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:57:59.193745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808afb3-f398-4c9a-b622-e177ac644a47 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:57:59.194020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808afb3-f398-4c9a-b622-e177ac644a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:57:59.198598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1117/2233] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 05:57:59 2022] PUT /identity/v3/projects/a3df8b054d4d4de5baa7778467f6e5a0/users/b1310c60f7624dfa83023cca199d5d30/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 05:57:59.212513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-1808afb3-f398-4c9a-b622-e177ac644a47 None None] MFA Rules not processed for user `e367c76dfa7f4d2791e60ce148f352c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:57:59.213744 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1808afb3-f398-4c9a-b622-e177ac644a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:57:59.279696 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1117/2234] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:57:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:58:03.010179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74940ca4-0356-44e6-8894-ce391f7d297f None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:58:03.010612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74940ca4-0356-44e6-8894-ce391f7d297f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:58:03.010708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74940ca4-0356-44e6-8894-ce391f7d297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:58:03.038667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-74940ca4-0356-44e6-8894-ce391f7d297f None None] MFA Rules not processed for user `e78b454f9d0b4e99ae0425cd596f02c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:58:03.039926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-74940ca4-0356-44e6-8894-ce391f7d297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:58:03.121990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1118/2235] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:58:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:58:04.282267 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eea00-a02c-4264-8af5-3117e7253129 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:58:04.282915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eea00-a02c-4264-8af5-3117e7253129 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:58:04.282915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eea00-a02c-4264-8af5-3117e7253129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:58:04.322863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-713eea00-a02c-4264-8af5-3117e7253129 None None] MFA Rules not processed for user `b1310c60f7624dfa83023cca199d5d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:58:04.325175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-713eea00-a02c-4264-8af5-3117e7253129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:58:04.434363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1118/2236] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:58:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:58:04.439893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74940ca4-0356-44e6-8894-ce391f7d297f None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:58:04.442005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-74940ca4-0356-44e6-8894-ce391f7d297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:58:04.530269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:58:04.564137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74940ca4-0356-44e6-8894-ce391f7d297f None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:58:04.581828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be982a0-aff7-4d88-9acd-8026775ae18c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:58:04.583125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be982a0-aff7-4d88-9acd-8026775ae18c service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:58:04.583485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be982a0-aff7-4d88-9acd-8026775ae18c service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:58:04.583937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be982a0-aff7-4d88-9acd-8026775ae18c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:58:04.657455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be982a0-aff7-4d88-9acd-8026775ae18c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:58:04.659140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be982a0-aff7-4d88-9acd-8026775ae18c service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:58:04.670936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1119/2237] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:58:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:58:14.421531 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=70 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:58:14.666450 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=34 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:58:23.783011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88024d-4de0-4b57-8a12-55e46f3f6514 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:58:23.784306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88024d-4de0-4b57-8a12-55e46f3f6514 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:58:23.784306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88024d-4de0-4b57-8a12-55e46f3f6514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:58:23.805327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ae88024d-4de0-4b57-8a12-55e46f3f6514 None None] MFA Rules not processed for user `982651a870c44a07bc4df322919e605e`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:58:23.806594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ae88024d-4de0-4b57-8a12-55e46f3f6514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:58:23.866693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1119/2238] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:58:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 05:58:24.659672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aee9f8-d754-4c73-9d1c-13e48ba7cf5b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:58:24.660231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aee9f8-d754-4c73-9d1c-13e48ba7cf5b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:58:24.660231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aee9f8-d754-4c73-9d1c-13e48ba7cf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:58:24.678125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-70aee9f8-d754-4c73-9d1c-13e48ba7cf5b None None] MFA Rules not processed for user `6ac9756c145040e58f9d914ba5bcdfaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:58:24.679582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-70aee9f8-d754-4c73-9d1c-13e48ba7cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:58:24.687969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1120/2239] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:58:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:58:33.860541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=25 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:58:34.673014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:58:54.617746 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88024d-4de0-4b57-8a12-55e46f3f6514 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:58:54.681099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88024d-4de0-4b57-8a12-55e46f3f6514 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:58:54.684741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232cec4-cefd-4a50-9267-3565b80fdab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:58:54.685393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232cec4-cefd-4a50-9267-3565b80fdab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:58:54.685672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232cec4-cefd-4a50-9267-3565b80fdab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:58:54.685935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232cec4-cefd-4a50-9267-3565b80fdab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:58:54.687744 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4232cec4-cefd-4a50-9267-3565b80fdab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:58:54.704134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232cec4-cefd-4a50-9267-3565b80fdab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:58:54.705630 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232cec4-cefd-4a50-9267-3565b80fdab0 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:58:54.725221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1120/2240] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:58:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:58:55.299265 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aee9f8-d754-4c73-9d1c-13e48ba7cf5b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:58:55.301605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-70aee9f8-d754-4c73-9d1c-13e48ba7cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:58:55.317551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aee9f8-d754-4c73-9d1c-13e48ba7cf5b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:58:55.321883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0418120-aa32-4249-b972-be2fdeb328ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:58:55.322559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0418120-aa32-4249-b972-be2fdeb328ed service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:58:55.322836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0418120-aa32-4249-b972-be2fdeb328ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:58:55.323108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0418120-aa32-4249-b972-be2fdeb328ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:58:55.325216 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f0418120-aa32-4249-b972-be2fdeb328ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:58:55.341351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0418120-aa32-4249-b972-be2fdeb328ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:58:55.342805 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0418120-aa32-4249-b972-be2fdeb328ed service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:58:55.354372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1121/2241] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:58:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:59:04.715182 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=26 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:59:04.723910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232cec4-cefd-4a50-9267-3565b80fdab0 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:59:04.738965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232cec4-cefd-4a50-9267-3565b80fdab0 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:59:04.748467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6e0f8-8394-4336-8420-386b872467b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:59:04.749113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6e0f8-8394-4336-8420-386b872467b7 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:59:04.749411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6e0f8-8394-4336-8420-386b872467b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:59:04.749681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6e0f8-8394-4336-8420-386b872467b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:59:04.751926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-67c6e0f8-8394-4336-8420-386b872467b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:59:04.763093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6e0f8-8394-4336-8420-386b872467b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:59:04.764803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6e0f8-8394-4336-8420-386b872467b7 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:59:04.774035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1121/2242] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:59:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:59:05.348957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:59:12.379715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0418120-aa32-4249-b972-be2fdeb328ed service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:59:12.391839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0418120-aa32-4249-b972-be2fdeb328ed service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:59:12.398956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d3681-0922-48d1-a0e4-01135e67e1da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:59:12.399670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d3681-0922-48d1-a0e4-01135e67e1da service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:59:12.399951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d3681-0922-48d1-a0e4-01135e67e1da service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:59:12.400231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d3681-0922-48d1-a0e4-01135e67e1da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:59:12.402254 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4f4d3681-0922-48d1-a0e4-01135e67e1da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:59:12.462184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d3681-0922-48d1-a0e4-01135e67e1da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:59:12.463962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d3681-0922-48d1-a0e4-01135e67e1da service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:59:12.473840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1122/2243] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:59:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:59:12.598023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6e0f8-8394-4336-8420-386b872467b7 service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:59:12.599494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-67c6e0f8-8394-4336-8420-386b872467b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:59:12.668253 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6e0f8-8394-4336-8420-386b872467b7 service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:59:12.672782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055ec2c-d72f-4310-a946-6b1a03d435fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:59:12.673466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ec2c-d72f-4310-a946-6b1a03d435fa service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:59:12.673733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ec2c-d72f-4310-a946-6b1a03d435fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:59:12.674002 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ec2c-d72f-4310-a946-6b1a03d435fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:59:12.676178 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1055ec2c-d72f-4310-a946-6b1a03d435fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:59:12.687898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055ec2c-d72f-4310-a946-6b1a03d435fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:59:12.690151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055ec2c-d72f-4310-a946-6b1a03d435fa service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:59:12.704363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1122/2244] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:59:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:59:22.469018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=28 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:59:22.482007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d3681-0922-48d1-a0e4-01135e67e1da service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:59:22.491835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d3681-0922-48d1-a0e4-01135e67e1da service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:59:22.497969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c72cd-acfa-45e7-980c-39e4b6bb531e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:59:22.498661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c72cd-acfa-45e7-980c-39e4b6bb531e service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:59:22.498941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c72cd-acfa-45e7-980c-39e4b6bb531e service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:59:22.504529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c72cd-acfa-45e7-980c-39e4b6bb531e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:59:22.506797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0d6c72cd-acfa-45e7-980c-39e4b6bb531e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:59:22.519219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c72cd-acfa-45e7-980c-39e4b6bb531e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:59:22.520829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c72cd-acfa-45e7-980c-39e4b6bb531e service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:59:22.530953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1123/2245] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:59:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:59:22.698100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=32 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:59:32.527017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:59:36.982439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73311d6d-9981-4516-958c-b53fc78d167d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:59:36.982951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73311d6d-9981-4516-958c-b53fc78d167d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:59:36.983053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73311d6d-9981-4516-958c-b53fc78d167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:59:37.003483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-73311d6d-9981-4516-958c-b53fc78d167d None None] MFA Rules not processed for user `93ae888c6e364e2e96ff52e63415a0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 05:59:37.004818 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-73311d6d-9981-4516-958c-b53fc78d167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:59:37.060595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1123/2246] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 05:59:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 05:59:42.049860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c72cd-acfa-45e7-980c-39e4b6bb531e service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 05:59:42.068051 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c72cd-acfa-45e7-980c-39e4b6bb531e service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 05:59:42.075484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba2549-7d1a-4386-8429-4ee4d21dedb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 05:59:42.075863 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba2549-7d1a-4386-8429-4ee4d21dedb1 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 05:59:42.076145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba2549-7d1a-4386-8429-4ee4d21dedb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 05:59:42.076689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba2549-7d1a-4386-8429-4ee4d21dedb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 05:59:42.079048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c5ba2549-7d1a-4386-8429-4ee4d21dedb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 05:59:42.093059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba2549-7d1a-4386-8429-4ee4d21dedb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 05:59:42.094669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba2549-7d1a-4386-8429-4ee4d21dedb1 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 05:59:42.107408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1124/2247] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 05:59:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 05:59:47.053251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=25 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 05:59:52.102493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:00:08.015703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400abe6-b552-4593-b605-47ecbada1de5 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:00:08.016288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400abe6-b552-4593-b605-47ecbada1de5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:00:08.016288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400abe6-b552-4593-b605-47ecbada1de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:00:08.034226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b400abe6-b552-4593-b605-47ecbada1de5 None None] MFA Rules not processed for user `32703b5451cf48b09966a7412b747099`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:00:08.035462 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b400abe6-b552-4593-b605-47ecbada1de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:00:08.042924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1124/2248] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:00:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:00:08.358899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba2549-7d1a-4386-8429-4ee4d21dedb1 service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:00:08.368980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba2549-7d1a-4386-8429-4ee4d21dedb1 service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:00:08.373998 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59dc67-40de-4109-b224-2d87612bbe82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:00:08.374634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59dc67-40de-4109-b224-2d87612bbe82 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:00:08.375002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59dc67-40de-4109-b224-2d87612bbe82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:00:08.375259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59dc67-40de-4109-b224-2d87612bbe82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:00:08.377192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2a59dc67-40de-4109-b224-2d87612bbe82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:00:08.387997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59dc67-40de-4109-b224-2d87612bbe82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:00:08.390252 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59dc67-40de-4109-b224-2d87612bbe82 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:00:08.400202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1125/2249] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:00:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:00:08.983282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400abe6-b552-4593-b605-47ecbada1de5 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:00:08.984800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b400abe6-b552-4593-b605-47ecbada1de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:00:09.057369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400abe6-b552-4593-b605-47ecbada1de5 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:00:09.062554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd99a1-c549-4a97-9a71-abece4accbb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:00:09.063212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd99a1-c549-4a97-9a71-abece4accbb4 service placement] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:00:09.063543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd99a1-c549-4a97-9a71-abece4accbb4 service placement] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:00:09.063822 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd99a1-c549-4a97-9a71-abece4accbb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:00:09.065628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-eedd99a1-c549-4a97-9a71-abece4accbb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:00:09.077110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd99a1-c549-4a97-9a71-abece4accbb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:00:09.078716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd99a1-c549-4a97-9a71-abece4accbb4 service placement] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:00:09.088501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1125/2250] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:00:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:00:18.237871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59dc67-40de-4109-b224-2d87612bbe82 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:00:18.237871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2a59dc67-40de-4109-b224-2d87612bbe82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:00:18.256099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59dc67-40de-4109-b224-2d87612bbe82 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:00:18.264584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a3bef-489a-4ab0-bd25-957ad17032a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:00:18.265486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a3bef-489a-4ab0-bd25-957ad17032a3 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:00:18.265959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a3bef-489a-4ab0-bd25-957ad17032a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:00:18.266357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a3bef-489a-4ab0-bd25-957ad17032a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:00:18.268441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7a7a3bef-489a-4ab0-bd25-957ad17032a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:00:18.284408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a3bef-489a-4ab0-bd25-957ad17032a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:00:18.286155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a3bef-489a-4ab0-bd25-957ad17032a3 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:00:18.299277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1126/2251] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:00:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:00:19.084452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=35 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:00:21.874106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd99a1-c549-4a97-9a71-abece4accbb4 service placement] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:00:21.887618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd99a1-c549-4a97-9a71-abece4accbb4 service placement] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:00:21.893810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c188614-d24a-43ce-bf97-fe6b20564dd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:00:21.894481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c188614-d24a-43ce-bf97-fe6b20564dd0 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:00:21.894840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c188614-d24a-43ce-bf97-fe6b20564dd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:00:21.895120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c188614-d24a-43ce-bf97-fe6b20564dd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:00:21.897266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0c188614-d24a-43ce-bf97-fe6b20564dd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:00:21.910644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c188614-d24a-43ce-bf97-fe6b20564dd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:00:21.912660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c188614-d24a-43ce-bf97-fe6b20564dd0 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:00:21.924442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1126/2252] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:00:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:00:28.294591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=12 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:00:31.919425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=6 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:00:35.356550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a3bef-489a-4ab0-bd25-957ad17032a3 service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:00:35.371282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a3bef-489a-4ab0-bd25-957ad17032a3 service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:00:35.378303 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca2e7-70df-4c2d-a7ec-98c8b5ad8d8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:00:35.378995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca2e7-70df-4c2d-a7ec-98c8b5ad8d8a service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:00:35.379317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca2e7-70df-4c2d-a7ec-98c8b5ad8d8a service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:00:35.379588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca2e7-70df-4c2d-a7ec-98c8b5ad8d8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:00:35.381759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f7bca2e7-70df-4c2d-a7ec-98c8b5ad8d8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:00:35.392523 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bca2e7-70df-4c2d-a7ec-98c8b5ad8d8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:00:35.394169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bca2e7-70df-4c2d-a7ec-98c8b5ad8d8a service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:00:35.403642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1127/2253] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:00:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:00:45.399630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:01:23.601844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c188614-d24a-43ce-bf97-fe6b20564dd0 service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:01:23.612322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c188614-d24a-43ce-bf97-fe6b20564dd0 service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:01:23.617043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6eac28-a28f-497a-952a-391b7f731aed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:01:23.617695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eac28-a28f-497a-952a-391b7f731aed service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:01:23.617979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eac28-a28f-497a-952a-391b7f731aed service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:01:23.618251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eac28-a28f-497a-952a-391b7f731aed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:01:23.620338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0e6eac28-a28f-497a-952a-391b7f731aed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:01:23.631541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6eac28-a28f-497a-952a-391b7f731aed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:01:23.632964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6eac28-a28f-497a-952a-391b7f731aed service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:01:23.642151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1127/2254] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:01:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:01:33.638293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=6 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:02:12.158432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca2e7-70df-4c2d-a7ec-98c8b5ad8d8a service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:02:12.160814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f7bca2e7-70df-4c2d-a7ec-98c8b5ad8d8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:02:12.179678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca2e7-70df-4c2d-a7ec-98c8b5ad8d8a service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:02:12.185581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa499d-681c-4050-8081-2555416a82e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:02:12.186239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa499d-681c-4050-8081-2555416a82e1 service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:02:12.186518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa499d-681c-4050-8081-2555416a82e1 service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:02:12.186827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa499d-681c-4050-8081-2555416a82e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:02:12.189078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-39fa499d-681c-4050-8081-2555416a82e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:02:12.201359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa499d-681c-4050-8081-2555416a82e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:02:12.202946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa499d-681c-4050-8081-2555416a82e1 service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:02:12.213957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1128/2255] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:02:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:02:22.209025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:03:03.733197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6eac28-a28f-497a-952a-391b7f731aed service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:03:03.754808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6eac28-a28f-497a-952a-391b7f731aed service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:03:03.761964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caf10d-1736-4ee7-aa64-c25ffc61166e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:03:03.762603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf10d-1736-4ee7-aa64-c25ffc61166e service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:03:03.762880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf10d-1736-4ee7-aa64-c25ffc61166e service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:03:03.763154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf10d-1736-4ee7-aa64-c25ffc61166e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:03:03.772311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63caf10d-1736-4ee7-aa64-c25ffc61166e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:03:03.773911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63caf10d-1736-4ee7-aa64-c25ffc61166e service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:03:03.783989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1128/2256] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:03:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:03:13.779146 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=6 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:03:56.065737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa499d-681c-4050-8081-2555416a82e1 service placement] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:03:56.076348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa499d-681c-4050-8081-2555416a82e1 service placement] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:03:56.080569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1694424-2c8b-4e44-9566-2a72a74d7794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:03:56.081202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1694424-2c8b-4e44-9566-2a72a74d7794 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:03:56.081474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1694424-2c8b-4e44-9566-2a72a74d7794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:03:56.081751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1694424-2c8b-4e44-9566-2a72a74d7794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:03:56.092307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1694424-2c8b-4e44-9566-2a72a74d7794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:03:56.093748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1694424-2c8b-4e44-9566-2a72a74d7794 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:03:56.107923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1129/2257] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:03:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:04:04.596275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caf10d-1736-4ee7-aa64-c25ffc61166e service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:04:04.598397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-63caf10d-1736-4ee7-aa64-c25ffc61166e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:04:04.612334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caf10d-1736-4ee7-aa64-c25ffc61166e service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:04:04.617243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4bd60-b0d4-41c2-b68b-d07a2f75cd54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:04:04.617892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4bd60-b0d4-41c2-b68b-d07a2f75cd54 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:04:04.618172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4bd60-b0d4-41c2-b68b-d07a2f75cd54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:04:04.618446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4bd60-b0d4-41c2-b68b-d07a2f75cd54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:04:04.629222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4bd60-b0d4-41c2-b68b-d07a2f75cd54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:04:04.630702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4bd60-b0d4-41c2-b68b-d07a2f75cd54 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:04:04.640298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1129/2258] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:04:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:04:06.099827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:04:12.609691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1694424-2c8b-4e44-9566-2a72a74d7794 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:04:12.627415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1694424-2c8b-4e44-9566-2a72a74d7794 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:04:12.636099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832eab41-ad3d-4abe-a978-ef131b1640d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:04:12.636752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eab41-ad3d-4abe-a978-ef131b1640d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:04:12.637032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eab41-ad3d-4abe-a978-ef131b1640d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:04:12.637310 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eab41-ad3d-4abe-a978-ef131b1640d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:04:12.647471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832eab41-ad3d-4abe-a978-ef131b1640d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:04:12.648968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832eab41-ad3d-4abe-a978-ef131b1640d9 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:04:12.659102 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1130/2259] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:04:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:04:12.710710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4bd60-b0d4-41c2-b68b-d07a2f75cd54 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:04:12.719347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4bd60-b0d4-41c2-b68b-d07a2f75cd54 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:04:12.723136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46890e-6021-48cf-af47-b436392ef54c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:04:12.723824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46890e-6021-48cf-af47-b436392ef54c service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:04:12.724099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46890e-6021-48cf-af47-b436392ef54c service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:04:12.724372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46890e-6021-48cf-af47-b436392ef54c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:04:12.733107 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46890e-6021-48cf-af47-b436392ef54c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:04:12.734548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46890e-6021-48cf-af47-b436392ef54c service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:04:12.744258 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1130/2260] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:04:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:04:22.660870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:04:22.739637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=12 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:04:26.358075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b23697-9def-46b5-9a62-f8a872881e43 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:04:26.358589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b23697-9def-46b5-9a62-f8a872881e43 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:04:26.358589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b23697-9def-46b5-9a62-f8a872881e43 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:04:26.360209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1131/2261] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jul 4 06:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 06:04:26.365450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda88f0-c38f-491c-bc97-225ebc8842c4 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:04:26.365933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda88f0-c38f-491c-bc97-225ebc8842c4 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:04:26.366044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda88f0-c38f-491c-bc97-225ebc8842c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:04:26.405982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-edda88f0-c38f-491c-bc97-225ebc8842c4 None None] MFA Rules not processed for user `486de932d1db422ba2c3d84142f9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:04:26.407193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-edda88f0-c38f-491c-bc97-225ebc8842c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:04:26.415819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1131/2262] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 06:04:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:04:26.438293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b23697-9def-46b5-9a62-f8a872881e43 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:04:26.460808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b23697-9def-46b5-9a62-f8a872881e43 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:04:26.467505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481f809-343a-4db6-9afc-736ffbd9da24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:04:26.468144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481f809-343a-4db6-9afc-736ffbd9da24 service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:04:26.468421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481f809-343a-4db6-9afc-736ffbd9da24 service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:04:26.468689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481f809-343a-4db6-9afc-736ffbd9da24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:04:26.477636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481f809-343a-4db6-9afc-736ffbd9da24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:04:26.479230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481f809-343a-4db6-9afc-736ffbd9da24 service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:04:26.489626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1132/2263] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:04:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:04:30.829209 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda88f0-c38f-491c-bc97-225ebc8842c4 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:04:30.840045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda88f0-c38f-491c-bc97-225ebc8842c4 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:04:30.844255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1882f10-3ed3-4b90-91b6-6caff0c29e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:04:30.844893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1882f10-3ed3-4b90-91b6-6caff0c29e5f service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:04:30.845172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1882f10-3ed3-4b90-91b6-6caff0c29e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:04:30.845436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1882f10-3ed3-4b90-91b6-6caff0c29e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:04:30.856916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1882f10-3ed3-4b90-91b6-6caff0c29e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:04:30.858363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1882f10-3ed3-4b90-91b6-6caff0c29e5f service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:04:30.869589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1132/2264] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:04:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:04:36.485368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:04:40.865226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=15 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:05:00.902577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481f809-343a-4db6-9afc-736ffbd9da24 service placement] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:05:00.920317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481f809-343a-4db6-9afc-736ffbd9da24 service placement] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:05:00.926532 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49798045-b425-442d-9cdf-4482eec61b0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:05:00.927347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49798045-b425-442d-9cdf-4482eec61b0b service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:05:00.927786 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49798045-b425-442d-9cdf-4482eec61b0b service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:05:00.928231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49798045-b425-442d-9cdf-4482eec61b0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:05:00.939727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49798045-b425-442d-9cdf-4482eec61b0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:05:00.941473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49798045-b425-442d-9cdf-4482eec61b0b service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:05:00.957690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1133/2265] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:05:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:05:03.695342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46b6d8-22d3-4daf-b5c9-f2f883454559 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:05:03.695885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46b6d8-22d3-4daf-b5c9-f2f883454559 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:05:03.695885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46b6d8-22d3-4daf-b5c9-f2f883454559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:05:03.746149 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6d46b6d8-22d3-4daf-b5c9-f2f883454559 None None] MFA Rules not processed for user `e367c76dfa7f4d2791e60ce148f352c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:05:03.747396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6d46b6d8-22d3-4daf-b5c9-f2f883454559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:05:03.764007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1133/2266] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:05:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:05:04.086639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49798045-b425-442d-9cdf-4482eec61b0b service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:05:04.096874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49798045-b425-442d-9cdf-4482eec61b0b service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:05:04.100983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63096f87-2733-4ae6-9139-bfe54c7e701d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:05:04.101667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63096f87-2733-4ae6-9139-bfe54c7e701d service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:05:04.101953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63096f87-2733-4ae6-9139-bfe54c7e701d service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:05:04.102232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63096f87-2733-4ae6-9139-bfe54c7e701d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:05:04.104154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-63096f87-2733-4ae6-9139-bfe54c7e701d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:05:04.113724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63096f87-2733-4ae6-9139-bfe54c7e701d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:05:04.115154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63096f87-2733-4ae6-9139-bfe54c7e701d service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:05:04.123031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1134/2267] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:05:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:05:06.114021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46b6d8-22d3-4daf-b5c9-f2f883454559 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:05:06.134216 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46b6d8-22d3-4daf-b5c9-f2f883454559 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:05:06.138398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0b0ab-b4f3-48ff-b05d-07af13660981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:05:06.139124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0b0ab-b4f3-48ff-b05d-07af13660981 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:05:06.139444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0b0ab-b4f3-48ff-b05d-07af13660981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:05:06.139720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0b0ab-b4f3-48ff-b05d-07af13660981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:05:06.149740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0b0ab-b4f3-48ff-b05d-07af13660981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:05:06.151201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0b0ab-b4f3-48ff-b05d-07af13660981 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:05:06.161392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1134/2268] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:05:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:05:14.119613 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=12 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:05:16.156947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=25 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:05:35.046685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63096f87-2733-4ae6-9139-bfe54c7e701d service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:05:35.062217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63096f87-2733-4ae6-9139-bfe54c7e701d service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:05:35.068881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c412710-aa66-41be-9815-f4a4669e8c0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:05:35.069542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c412710-aa66-41be-9815-f4a4669e8c0f service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:05:35.069828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c412710-aa66-41be-9815-f4a4669e8c0f service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:05:35.070106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c412710-aa66-41be-9815-f4a4669e8c0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:05:35.078460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c412710-aa66-41be-9815-f4a4669e8c0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:05:35.080093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c412710-aa66-41be-9815-f4a4669e8c0f service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:05:35.107642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1135/2269] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:05:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:05:45.104815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=13 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:06:22.855588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0b0ab-b4f3-48ff-b05d-07af13660981 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:06:22.869764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0b0ab-b4f3-48ff-b05d-07af13660981 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:06:22.876049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4d782-1b0b-4556-bc8b-885aaa0abaa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:06:22.876981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4d782-1b0b-4556-bc8b-885aaa0abaa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:06:22.877358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4d782-1b0b-4556-bc8b-885aaa0abaa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:06:22.877715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4d782-1b0b-4556-bc8b-885aaa0abaa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:06:22.887570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4d782-1b0b-4556-bc8b-885aaa0abaa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:06:22.888994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4d782-1b0b-4556-bc8b-885aaa0abaa3 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:06:22.917400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1135/2270] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:06:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:06:32.914642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=13 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:07:15.774674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c412710-aa66-41be-9815-f4a4669e8c0f service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:07:15.840409 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c412710-aa66-41be-9815-f4a4669e8c0f service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:07:15.846480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf7163-530c-4287-bd15-66f63dd5320f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:07:15.847130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf7163-530c-4287-bd15-66f63dd5320f service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:07:15.847465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf7163-530c-4287-bd15-66f63dd5320f service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:07:15.847738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf7163-530c-4287-bd15-66f63dd5320f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:07:15.857242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf7163-530c-4287-bd15-66f63dd5320f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:07:15.858828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf7163-530c-4287-bd15-66f63dd5320f service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:07:15.869214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1136/2271] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:07:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:07:25.864924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=27 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:08:03.867580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4d782-1b0b-4556-bc8b-885aaa0abaa3 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:08:03.905489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4d782-1b0b-4556-bc8b-885aaa0abaa3 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:08:03.911004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e7d5e-b7b8-4a7c-a471-751f27835aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:08:03.911693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e7d5e-b7b8-4a7c-a471-751f27835aae service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:08:03.911969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e7d5e-b7b8-4a7c-a471-751f27835aae service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:08:03.912236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e7d5e-b7b8-4a7c-a471-751f27835aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:08:03.914263 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ed9e7d5e-b7b8-4a7c-a471-751f27835aae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:08:03.956498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e7d5e-b7b8-4a7c-a471-751f27835aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:08:03.957929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e7d5e-b7b8-4a7c-a471-751f27835aae service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:08:03.976328 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1136/2272] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:08:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:08:13.973787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=32 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:09:05.958110 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf7163-530c-4287-bd15-66f63dd5320f service placement] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:09:05.968265 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf7163-530c-4287-bd15-66f63dd5320f service placement] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:09:05.972596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d0746-87e6-40f7-837d-060524b39924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:09:05.973246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d0746-87e6-40f7-837d-060524b39924 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:09:05.973529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d0746-87e6-40f7-837d-060524b39924 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:09:05.973808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d0746-87e6-40f7-837d-060524b39924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:09:05.976033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4f4d0746-87e6-40f7-837d-060524b39924 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:09:06.059196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d0746-87e6-40f7-837d-060524b39924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:09:06.060675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d0746-87e6-40f7-837d-060524b39924 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:09:06.079455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1137/2273] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:09:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:09:13.366678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e7d5e-b7b8-4a7c-a471-751f27835aae service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:09:13.428022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e7d5e-b7b8-4a7c-a471-751f27835aae service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:09:13.432714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc44886-c4c0-438e-948b-2aaf8b982591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:09:13.433355 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc44886-c4c0-438e-948b-2aaf8b982591 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:09:13.433756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc44886-c4c0-438e-948b-2aaf8b982591 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:09:13.434023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc44886-c4c0-438e-948b-2aaf8b982591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:09:13.436275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-fbc44886-c4c0-438e-948b-2aaf8b982591 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:09:13.522645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc44886-c4c0-438e-948b-2aaf8b982591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:09:13.524132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc44886-c4c0-438e-948b-2aaf8b982591 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:09:13.544930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1137/2274] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:09:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:09:13.634503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d0746-87e6-40f7-837d-060524b39924 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:09:13.689841 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d0746-87e6-40f7-837d-060524b39924 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:09:13.693667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f085e-a44a-4795-974a-32980dba4aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:09:13.694323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f085e-a44a-4795-974a-32980dba4aff service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:09:13.694603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f085e-a44a-4795-974a-32980dba4aff service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:09:13.694862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f085e-a44a-4795-974a-32980dba4aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:09:13.696850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d58f085e-a44a-4795-974a-32980dba4aff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:09:13.768697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f085e-a44a-4795-974a-32980dba4aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:09:13.770159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f085e-a44a-4795-974a-32980dba4aff service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:09:13.779479 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1138/2275] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:09:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:09:23.541633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=61 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:09:23.775569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:10:17.832073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc44886-c4c0-438e-948b-2aaf8b982591 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:10:17.834223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-fbc44886-c4c0-438e-948b-2aaf8b982591 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:17.928828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc44886-c4c0-438e-948b-2aaf8b982591 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:10:17.933523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7855bd-54e1-42ed-906e-12a8f1560a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:10:17.934172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7855bd-54e1-42ed-906e-12a8f1560a13 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:17.934453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7855bd-54e1-42ed-906e-12a8f1560a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:17.934729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7855bd-54e1-42ed-906e-12a8f1560a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:17.936820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cf7855bd-54e1-42ed-906e-12a8f1560a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:17.946619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7855bd-54e1-42ed-906e-12a8f1560a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:10:17.948119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7855bd-54e1-42ed-906e-12a8f1560a13 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:10:17.956450 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1138/2276] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:10:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:10:27.947547 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f085e-a44a-4795-974a-32980dba4aff service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:10:27.952721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=35 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:10:27.957506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f085e-a44a-4795-974a-32980dba4aff service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:10:27.961365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e708f0-81b4-4d02-85fd-17cf0586f533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:10:27.962033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e708f0-81b4-4d02-85fd-17cf0586f533 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:27.962316 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e708f0-81b4-4d02-85fd-17cf0586f533 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:27.962593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e708f0-81b4-4d02-85fd-17cf0586f533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:27.964560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c6e708f0-81b4-4d02-85fd-17cf0586f533 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:28.050573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e708f0-81b4-4d02-85fd-17cf0586f533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:10:28.052075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e708f0-81b4-4d02-85fd-17cf0586f533 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:10:28.074465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1139/2277] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:10:30.987897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd0da-d47b-4322-bba3-e24c659e3e11 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:30.987897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd0da-d47b-4322-bba3-e24c659e3e11 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:30.987897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfd0da-d47b-4322-bba3-e24c659e3e11 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:30.987897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1139/2278] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 06:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 06:10:30.993867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fcba1-ba8a-4793-a149-f6a33b490ddb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:30.994798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fcba1-ba8a-4793-a149-f6a33b490ddb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:30.995706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fcba1-ba8a-4793-a149-f6a33b490ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:31.063896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-0d8fcba1-ba8a-4793-a149-f6a33b490ddb None None] MFA Rules not processed for user `486de932d1db422ba2c3d84142f9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:10:31.063896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0d8fcba1-ba8a-4793-a149-f6a33b490ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:31.130779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1140/2279] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 06:10:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:10:31.142161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfd0da-d47b-4322-bba3-e24c659e3e11 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:10:31.156783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfd0da-d47b-4322-bba3-e24c659e3e11 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:10:31.161984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455793bb-8b4d-4630-88b2-dc9cab5c278b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:10:31.162630 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455793bb-8b4d-4630-88b2-dc9cab5c278b service placement] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:31.162906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455793bb-8b4d-4630-88b2-dc9cab5c278b service placement] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:31.163177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455793bb-8b4d-4630-88b2-dc9cab5c278b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:31.171785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455793bb-8b4d-4630-88b2-dc9cab5c278b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:10:31.173494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455793bb-8b4d-4630-88b2-dc9cab5c278b service placement] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:10:31.184399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1140/2280] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:10:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:10:32.211540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fcba1-ba8a-4793-a149-f6a33b490ddb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:10:32.213885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0d8fcba1-ba8a-4793-a149-f6a33b490ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:32.226273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fcba1-ba8a-4793-a149-f6a33b490ddb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:10:32.230959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668abec6-d996-49cd-be55-85d9bed8cc69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:10:32.231923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668abec6-d996-49cd-be55-85d9bed8cc69 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:32.232395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668abec6-d996-49cd-be55-85d9bed8cc69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:32.232844 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668abec6-d996-49cd-be55-85d9bed8cc69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:32.235031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-668abec6-d996-49cd-be55-85d9bed8cc69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:32.271492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:10:32.329938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668abec6-d996-49cd-be55-85d9bed8cc69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:10:32.331440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668abec6-d996-49cd-be55-85d9bed8cc69 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:10:32.354535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1141/2281] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:10:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:10:33.833584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455793bb-8b4d-4630-88b2-dc9cab5c278b service placement] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:10:33.842879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455793bb-8b4d-4630-88b2-dc9cab5c278b service placement] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:10:33.846459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767ebb2-4f04-4992-b541-f21dfb53e2ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:10:33.847102 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767ebb2-4f04-4992-b541-f21dfb53e2ce service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:33.847425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767ebb2-4f04-4992-b541-f21dfb53e2ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:33.847704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767ebb2-4f04-4992-b541-f21dfb53e2ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:33.849654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2767ebb2-4f04-4992-b541-f21dfb53e2ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:33.863391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767ebb2-4f04-4992-b541-f21dfb53e2ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:10:33.865090 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767ebb2-4f04-4992-b541-f21dfb53e2ce service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:10:33.876042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1141/2282] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:10:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:10:34.909147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668abec6-d996-49cd-be55-85d9bed8cc69 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:10:34.911073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-668abec6-d996-49cd-be55-85d9bed8cc69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:34.928215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668abec6-d996-49cd-be55-85d9bed8cc69 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:10:34.933496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:10:34.934396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:34.934675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:34.934946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] PATH_INFO: `/v3/users/6ac9756c145040e58f9d914ba5bcdfaa` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:34.936419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac9756c145040e58f9d914ba5bcdfaa)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:10:34.938354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:10:34.998749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac9756c145040e58f9d914ba5bcdfaa'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.008334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac9756c145040e58f9d914ba5bcdfaa'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.018132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac9756c145040e58f9d914ba5bcdfaa'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.018821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac9756c145040e58f9d914ba5bcdfaa'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.019491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac9756c145040e58f9d914ba5bcdfaa'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.037498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1142/2283] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:10:34 2022] DELETE /identity/v3/users/6ac9756c145040e58f9d914ba5bcdfaa => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:10:35.129704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767ebb2-4f04-4992-b541-f21dfb53e2ce service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:10:35.141373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767ebb2-4f04-4992-b541-f21dfb53e2ce service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:10:35.197097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a58613-db8a-4c42-973b-b91c9697dd09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:10:35.197763 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a58613-db8a-4c42-973b-b91c9697dd09 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:35.198045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a58613-db8a-4c42-973b-b91c9697dd09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:35.198315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a58613-db8a-4c42-973b-b91c9697dd09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:35.200156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-85a58613-db8a-4c42-973b-b91c9697dd09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:35.222000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:10:35.223760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:35.294272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a58613-db8a-4c42-973b-b91c9697dd09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:10:35.299373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a58613-db8a-4c42-973b-b91c9697dd09 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:10:35.300440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82196f5a-254e-4926-9085-ab4bc173621d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:10:35.305294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:10:35.305802 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1142/2284] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:10:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:10:35.306466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:35.306762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:35.307002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] PATH_INFO: `/v3/projects/8d701a66ecbb480fa08e97f304acf57b` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:35.308205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d701a66ecbb480fa08e97f304acf57b)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:10:35.310569 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:10:35.318140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:10:35.373330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d701a66ecbb480fa08e97f304acf57b', 'request_id': 'req-00f4154a-d5c7-479d-b81f-30f16a4ce49a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.379682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d701a66ecbb480fa08e97f304acf57b', 'request_id': 'req-00f4154a-d5c7-479d-b81f-30f16a4ce49a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.385323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d701a66ecbb480fa08e97f304acf57b', 'request_id': 'req-00f4154a-d5c7-479d-b81f-30f16a4ce49a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.402128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] The token cache is being invalidate because project 8d701a66ecbb480fa08e97f304acf57b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:10:35.402414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.403046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.404770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1143/2285] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:10:35 2022] DELETE /identity/v3/projects/8d701a66ecbb480fa08e97f304acf57b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:10:35.409330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a58613-db8a-4c42-973b-b91c9697dd09 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:10:35.421048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a58613-db8a-4c42-973b-b91c9697dd09 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:10:35.479676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:10:35.480593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:35.480871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:35.481153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] PATH_INFO: `/v3/users/b1310c60f7624dfa83023cca199d5d30` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:35.494295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1310c60f7624dfa83023cca199d5d30)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:10:35.496291 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:10:35.501759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:10:35.546133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1310c60f7624dfa83023cca199d5d30'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.553135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1310c60f7624dfa83023cca199d5d30'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.553808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1310c60f7624dfa83023cca199d5d30'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.563017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1310c60f7624dfa83023cca199d5d30'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.575656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1310c60f7624dfa83023cca199d5d30'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.581864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1143/2286] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:10:35 2022] DELETE /identity/v3/users/b1310c60f7624dfa83023cca199d5d30 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:10:35.764443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:10:35.766052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:10:35.784129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4154a-d5c7-479d-b81f-30f16a4ce49a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:10:35.843651 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:10:35.844520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:10:35.844796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:10:35.845061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] PATH_INFO: `/v3/projects/a3df8b054d4d4de5baa7778467f6e5a0` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:10:35.855394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3df8b054d4d4de5baa7778467f6e5a0)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:10:35.857314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:10:35.910884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3df8b054d4d4de5baa7778467f6e5a0', 'request_id': 'req-7e8b6cc3-9380-48e0-950f-82eee6197d78'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.917520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3df8b054d4d4de5baa7778467f6e5a0', 'request_id': 'req-7e8b6cc3-9380-48e0-950f-82eee6197d78'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.923287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3df8b054d4d4de5baa7778467f6e5a0', 'request_id': 'req-7e8b6cc3-9380-48e0-950f-82eee6197d78'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.937909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] The token cache is being invalidate because project a3df8b054d4d4de5baa7778467f6e5a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:10:35.938111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.938810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:10:35.940554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1144/2287] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:10:35 2022] DELETE /identity/v3/projects/a3df8b054d4d4de5baa7778467f6e5a0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:10:45.578360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:10:45.934714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=77,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:11:22.476297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:11:22.477878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:11:22.540880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5071a9-1206-426c-b8da-929673754ede admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:11:22.594898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e06a4-d302-4bf0-b2d8-c7216f8e91e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:11:22.595599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e06a4-d302-4bf0-b2d8-c7216f8e91e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:11:22.595882 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e06a4-d302-4bf0-b2d8-c7216f8e91e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:11:22.596153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e06a4-d302-4bf0-b2d8-c7216f8e91e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:11:22.599160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1c1e06a4-d302-4bf0-b2d8-c7216f8e91e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:11:22.689005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e06a4-d302-4bf0-b2d8-c7216f8e91e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:11:22.690876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e06a4-d302-4bf0-b2d8-c7216f8e91e8 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:11:22.705472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1144/2288] 10.222.0.42 () {60 vars in 1396 bytes} [Mon Jul 4 06:11:22 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3248 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:11:32.698392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=73 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:11:52.311926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:11:52.313366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:11:52.442272 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b6cc3-9380-48e0-950f-82eee6197d78 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:11:52.450058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02758e61-b21d-48c9-b406-a1c742ffc1b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:11:52.450892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02758e61-b21d-48c9-b406-a1c742ffc1b3 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:11:52.451360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02758e61-b21d-48c9-b406-a1c742ffc1b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:11:52.451796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02758e61-b21d-48c9-b406-a1c742ffc1b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:11:52.454064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-02758e61-b21d-48c9-b406-a1c742ffc1b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:11:52.469041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02758e61-b21d-48c9-b406-a1c742ffc1b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:11:52.470719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02758e61-b21d-48c9-b406-a1c742ffc1b3 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:11:52.483011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1145/2289] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:11:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:12:02.478761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=34 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:12:18.128234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e06a4-d302-4bf0-b2d8-c7216f8e91e8 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:12:18.129989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1c1e06a4-d302-4bf0-b2d8-c7216f8e91e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:12:18.212299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e06a4-d302-4bf0-b2d8-c7216f8e91e8 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:12:18.219397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d77f-031b-45c8-878d-91eec5389c22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:12:18.220058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d77f-031b-45c8-878d-91eec5389c22 service placement] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:12:18.220337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d77f-031b-45c8-878d-91eec5389c22 service placement] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:12:18.220609 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d77f-031b-45c8-878d-91eec5389c22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:12:18.222944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-bb61d77f-031b-45c8-878d-91eec5389c22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:12:18.239292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61d77f-031b-45c8-878d-91eec5389c22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:12:18.241005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61d77f-031b-45c8-878d-91eec5389c22 service placement] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:12:18.251932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1145/2290] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:12:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:12:28.247749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=32 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:14:22.344823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02758e61-b21d-48c9-b406-a1c742ffc1b3 service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:14:22.347227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-02758e61-b21d-48c9-b406-a1c742ffc1b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:14:22.418617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02758e61-b21d-48c9-b406-a1c742ffc1b3 service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:14:22.423581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a65752-1313-48f0-99eb-c2ce5c646cee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:14:22.424241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a65752-1313-48f0-99eb-c2ce5c646cee service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:14:22.424524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a65752-1313-48f0-99eb-c2ce5c646cee service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:14:22.424798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a65752-1313-48f0-99eb-c2ce5c646cee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:14:22.427064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a2a65752-1313-48f0-99eb-c2ce5c646cee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:14:22.491906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a65752-1313-48f0-99eb-c2ce5c646cee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:14:22.493361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a65752-1313-48f0-99eb-c2ce5c646cee service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:14:22.502714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1146/2291] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:14:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:14:22.570218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d77f-031b-45c8-878d-91eec5389c22 service placement] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:14:22.579931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d77f-031b-45c8-878d-91eec5389c22 service placement] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:14:22.591415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941840-f98c-45cb-9028-06c906a4c45a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:14:22.591970 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941840-f98c-45cb-9028-06c906a4c45a service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:14:22.592247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941840-f98c-45cb-9028-06c906a4c45a service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:14:22.592636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941840-f98c-45cb-9028-06c906a4c45a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:14:22.594775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4b941840-f98c-45cb-9028-06c906a4c45a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:14:22.663798 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941840-f98c-45cb-9028-06c906a4c45a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:14:22.665969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941840-f98c-45cb-9028-06c906a4c45a service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:14:22.678926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1146/2292] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:14:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:14:22.724171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a65752-1313-48f0-99eb-c2ce5c646cee service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:14:22.733106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a65752-1313-48f0-99eb-c2ce5c646cee service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:14:22.737827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c0b83-e7f9-4dfa-8fa7-322024a00c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:14:22.738472 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c0b83-e7f9-4dfa-8fa7-322024a00c2e service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:14:22.738755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c0b83-e7f9-4dfa-8fa7-322024a00c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:14:22.739027 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c0b83-e7f9-4dfa-8fa7-322024a00c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:14:22.741348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2c3c0b83-e7f9-4dfa-8fa7-322024a00c2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:14:22.812080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c0b83-e7f9-4dfa-8fa7-322024a00c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:14:22.813524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c0b83-e7f9-4dfa-8fa7-322024a00c2e service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:14:22.823927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1147/2293] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:14:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:14:32.672965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=28 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:14:32.818992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=81 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:15:23.824962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941840-f98c-45cb-9028-06c906a4c45a service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:15:23.826726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4b941840-f98c-45cb-9028-06c906a4c45a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:15:23.839860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941840-f98c-45cb-9028-06c906a4c45a service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:15:23.844369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21065194-7cf7-4132-afc5-58fb23559cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:15:23.845001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21065194-7cf7-4132-afc5-58fb23559cad service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:15:23.845285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21065194-7cf7-4132-afc5-58fb23559cad service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:15:23.845557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21065194-7cf7-4132-afc5-58fb23559cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:15:23.847497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-21065194-7cf7-4132-afc5-58fb23559cad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:15:23.858035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21065194-7cf7-4132-afc5-58fb23559cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:15:23.859506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21065194-7cf7-4132-afc5-58fb23559cad service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:15:23.867859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1147/2294] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:15:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:15:32.830046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c0b83-e7f9-4dfa-8fa7-322024a00c2e service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:15:32.832010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2c3c0b83-e7f9-4dfa-8fa7-322024a00c2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:15:32.845854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c0b83-e7f9-4dfa-8fa7-322024a00c2e service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:15:32.850468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33729c3d-77f8-4740-86b1-f5e3f4128d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:15:32.851159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33729c3d-77f8-4740-86b1-f5e3f4128d51 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:15:32.851475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33729c3d-77f8-4740-86b1-f5e3f4128d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:15:32.851756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33729c3d-77f8-4740-86b1-f5e3f4128d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:15:32.853915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-33729c3d-77f8-4740-86b1-f5e3f4128d51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:15:32.867950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33729c3d-77f8-4740-86b1-f5e3f4128d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:15:32.869387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33729c3d-77f8-4740-86b1-f5e3f4128d51 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:15:32.880475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1148/2295] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:15:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:15:33.864355 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=6 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:15:34.248112 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21065194-7cf7-4132-afc5-58fb23559cad service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:15:34.260849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21065194-7cf7-4132-afc5-58fb23559cad service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:15:34.267269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec96445-576e-464b-bd17-cc6531ef25da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:15:34.268089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec96445-576e-464b-bd17-cc6531ef25da service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:15:34.268408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec96445-576e-464b-bd17-cc6531ef25da service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:15:34.268760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec96445-576e-464b-bd17-cc6531ef25da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:15:34.271544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5ec96445-576e-464b-bd17-cc6531ef25da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:15:34.285562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec96445-576e-464b-bd17-cc6531ef25da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:15:34.287095 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec96445-576e-464b-bd17-cc6531ef25da service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:15:34.298054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1148/2296] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:15:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:15:36.463836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33729c3d-77f8-4740-86b1-f5e3f4128d51 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:15:36.465370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-33729c3d-77f8-4740-86b1-f5e3f4128d51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:15:36.479738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33729c3d-77f8-4740-86b1-f5e3f4128d51 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:15:36.486802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9331a5-c6e1-4e6e-97db-2ce3c2aaec42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:15:36.487504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9331a5-c6e1-4e6e-97db-2ce3c2aaec42 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:15:36.487788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9331a5-c6e1-4e6e-97db-2ce3c2aaec42 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:15:36.488032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9331a5-c6e1-4e6e-97db-2ce3c2aaec42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:15:36.490284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cd9331a5-c6e1-4e6e-97db-2ce3c2aaec42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:15:36.561887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9331a5-c6e1-4e6e-97db-2ce3c2aaec42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:15:36.563568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9331a5-c6e1-4e6e-97db-2ce3c2aaec42 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:15:36.582437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1149/2297] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:15:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:15:44.293418 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=6 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:15:46.579593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=42 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:16:23.660342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec96445-576e-464b-bd17-cc6531ef25da service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:16:23.679296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec96445-576e-464b-bd17-cc6531ef25da service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:16:23.688058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19388c-ee8e-4fdb-8ff7-8f6bb698b151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:16:23.688700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19388c-ee8e-4fdb-8ff7-8f6bb698b151 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:16:23.688990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19388c-ee8e-4fdb-8ff7-8f6bb698b151 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:16:23.689217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19388c-ee8e-4fdb-8ff7-8f6bb698b151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:16:23.697846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19388c-ee8e-4fdb-8ff7-8f6bb698b151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:16:23.699479 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19388c-ee8e-4fdb-8ff7-8f6bb698b151 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:16:23.727728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1149/2298] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:16:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:16:33.724985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=13 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:17:19.096095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9331a5-c6e1-4e6e-97db-2ce3c2aaec42 service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:17:19.098332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cd9331a5-c6e1-4e6e-97db-2ce3c2aaec42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:17:19.162946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9331a5-c6e1-4e6e-97db-2ce3c2aaec42 service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:17:19.169099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db404-8d7a-42d7-8bd8-20b3caf008aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:17:19.169756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db404-8d7a-42d7-8bd8-20b3caf008aa service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:17:19.170039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db404-8d7a-42d7-8bd8-20b3caf008aa service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:17:19.170318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db404-8d7a-42d7-8bd8-20b3caf008aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:17:19.179983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61db404-8d7a-42d7-8bd8-20b3caf008aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:17:19.181590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61db404-8d7a-42d7-8bd8-20b3caf008aa service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:17:19.192048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1150/2299] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:17:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:17:29.187830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=23 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:18:23.605949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19388c-ee8e-4fdb-8ff7-8f6bb698b151 service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:23.625953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19388c-ee8e-4fdb-8ff7-8f6bb698b151 service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:23.630020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bae32b-c016-469b-8e09-0954416cbf4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:23.630668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bae32b-c016-469b-8e09-0954416cbf4a service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:23.630950 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bae32b-c016-469b-8e09-0954416cbf4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:23.631221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bae32b-c016-469b-8e09-0954416cbf4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:23.633175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-82bae32b-c016-469b-8e09-0954416cbf4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:18:23.710685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bae32b-c016-469b-8e09-0954416cbf4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:23.712152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bae32b-c016-469b-8e09-0954416cbf4a service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:23.722097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1150/2300] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:18:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:18:26.589815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db404-8d7a-42d7-8bd8-20b3caf008aa service placement] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:26.615877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db404-8d7a-42d7-8bd8-20b3caf008aa service placement] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:26.621061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3af98-8c0a-41a8-a628-f3589160fea0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:26.621960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3af98-8c0a-41a8-a628-f3589160fea0 service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:26.622242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3af98-8c0a-41a8-a628-f3589160fea0 service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:26.622512 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3af98-8c0a-41a8-a628-f3589160fea0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:26.624360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e6c3af98-8c0a-41a8-a628-f3589160fea0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:18:26.643631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3af98-8c0a-41a8-a628-f3589160fea0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:26.645811 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3af98-8c0a-41a8-a628-f3589160fea0 service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:26.657393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1151/2301] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:18:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:18:26.663580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bae32b-c016-469b-8e09-0954416cbf4a service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:26.665760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-82bae32b-c016-469b-8e09-0954416cbf4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:18:26.681118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bae32b-c016-469b-8e09-0954416cbf4a service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:26.685906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:26.686796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:26.687073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:26.687375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] PATH_INFO: `/v3/users/982651a870c44a07bc4df322919e605e` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:26.688553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] RBAC: Authorizing `identity:delete_user(user_id=982651a870c44a07bc4df322919e605e)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:26.690469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:26.739174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982651a870c44a07bc4df322919e605e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:18:26.747070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982651a870c44a07bc4df322919e605e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:18:26.747871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982651a870c44a07bc4df322919e605e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:18:26.760466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982651a870c44a07bc4df322919e605e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:18:26.774466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eeed3a9b-aed4-457a-ad67-5eda72b4299f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982651a870c44a07bc4df322919e605e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:18:26.780952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1151/2302] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:18:26 2022] DELETE /identity/v3/users/982651a870c44a07bc4df322919e605e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:18:26.972700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3af98-8c0a-41a8-a628-f3589160fea0 service placement] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:26.984102 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3af98-8c0a-41a8-a628-f3589160fea0 service placement] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:27.049582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:27.050438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.050713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.050986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] PATH_INFO: `/v3/projects/f3251b70a6c64287acc95cc99d4e3b4d` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.052207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3251b70a6c64287acc95cc99d4e3b4d)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:27.054349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:27.103270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3251b70a6c64287acc95cc99d4e3b4d', 'request_id': 'req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:18:27.109602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3251b70a6c64287acc95cc99d4e3b4d', 'request_id': 'req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:18:27.115683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3251b70a6c64287acc95cc99d4e3b4d', 'request_id': 'req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:18:27.130653 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] The token cache is being invalidate because project f3251b70a6c64287acc95cc99d4e3b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:18:27.130854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:18:27.131525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d2b72e4a-6a3f-4ac8-a178-a8b2dec02c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:18:27.133128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1152/2303] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:18:26 2022] DELETE /identity/v3/projects/f3251b70a6c64287acc95cc99d4e3b4d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:18:27.140847 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561c45d-ac0d-4112-bc4f-c9383aa49867 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.141219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561c45d-ac0d-4112-bc4f-c9383aa49867 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.141595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561c45d-ac0d-4112-bc4f-c9383aa49867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.193252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c561c45d-ac0d-4112-bc4f-c9383aa49867 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:18:27.194481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c561c45d-ac0d-4112-bc4f-c9383aa49867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:18:27.214677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:18:27.257616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1152/2304] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:18:27.265942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c803cd-416d-4358-8c9a-00c201192501 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.266239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c803cd-416d-4358-8c9a-00c201192501 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.266498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c803cd-416d-4358-8c9a-00c201192501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.285742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-b8c803cd-416d-4358-8c9a-00c201192501 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:18:27.286954 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b8c803cd-416d-4358-8c9a-00c201192501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:18:27.295437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1153/2305] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:18:27.299774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561c45d-ac0d-4112-bc4f-c9383aa49867 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:27.310954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561c45d-ac0d-4112-bc4f-c9383aa49867 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:27.316583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e1288-bfcb-496c-9aaa-cd1b74eef308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:27.317250 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e1288-bfcb-496c-9aaa-cd1b74eef308 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.317535 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e1288-bfcb-496c-9aaa-cd1b74eef308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.317808 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e1288-bfcb-496c-9aaa-cd1b74eef308 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.318555 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e1288-bfcb-496c-9aaa-cd1b74eef308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:18:27.318863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e1288-bfcb-496c-9aaa-cd1b74eef308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:27.320507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e1288-bfcb-496c-9aaa-cd1b74eef308 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:27.336083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1153/2306] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:18:27.341033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c803cd-416d-4358-8c9a-00c201192501 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:27.354973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c803cd-416d-4358-8c9a-00c201192501 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:27.360676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf57d97-8645-4e01-9309-78afb243d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:27.361335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf57d97-8645-4e01-9309-78afb243d579 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.361618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf57d97-8645-4e01-9309-78afb243d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.361885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf57d97-8645-4e01-9309-78afb243d579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.362586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf57d97-8645-4e01-9309-78afb243d579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:27.364622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf57d97-8645-4e01-9309-78afb243d579 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:27.404180 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1154/2307] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:18:27 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:18:27.409024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e1288-bfcb-496c-9aaa-cd1b74eef308 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:27.421845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e1288-bfcb-496c-9aaa-cd1b74eef308 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:27.479899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc991899-5bb2-4b36-941f-7860b2bb3c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:27.480567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc991899-5bb2-4b36-941f-7860b2bb3c78 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.480845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc991899-5bb2-4b36-941f-7860b2bb3c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.481118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc991899-5bb2-4b36-941f-7860b2bb3c78 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.481828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc991899-5bb2-4b36-941f-7860b2bb3c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:27.483857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc991899-5bb2-4b36-941f-7860b2bb3c78 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:27.505788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1154/2308] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:18:27 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:18:27.510145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf57d97-8645-4e01-9309-78afb243d579 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:27.521015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf57d97-8645-4e01-9309-78afb243d579 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:27.526436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db762734-67ae-47e6-8783-cd64afd99ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:27.527115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762734-67ae-47e6-8783-cd64afd99ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.527431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762734-67ae-47e6-8783-cd64afd99ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.527706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762734-67ae-47e6-8783-cd64afd99ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.528450 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db762734-67ae-47e6-8783-cd64afd99ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:27.530104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db762734-67ae-47e6-8783-cd64afd99ab0 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:27.540611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1155/2309] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:18:27.546773 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc991899-5bb2-4b36-941f-7860b2bb3c78 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:27.562381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc991899-5bb2-4b36-941f-7860b2bb3c78 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:27.567824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d3df9-4d29-4317-b384-18c6d8141668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:27.568487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d3df9-4d29-4317-b384-18c6d8141668 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.568813 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d3df9-4d29-4317-b384-18c6d8141668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.569114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d3df9-4d29-4317-b384-18c6d8141668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.569829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d3df9-4d29-4317-b384-18c6d8141668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:27.571616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d3df9-4d29-4317-b384-18c6d8141668 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:27.583488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1155/2310] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:18:27.588231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db762734-67ae-47e6-8783-cd64afd99ab0 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:27.598926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db762734-67ae-47e6-8783-cd64afd99ab0 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:27.603823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277939a0-a417-4549-baa6-600ca4574d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:27.604717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277939a0-a417-4549-baa6-600ca4574d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.604993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277939a0-a417-4549-baa6-600ca4574d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.605265 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277939a0-a417-4549-baa6-600ca4574d53 admin admin] PATH_INFO: `/v3/projects/aac7de21b69441899866f7685df2e5c2/users/9a963453dec24163ada4a51322997303/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.621691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277939a0-a417-4549-baa6-600ca4574d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac7de21b69441899866f7685df2e5c2, user_id=9a963453dec24163ada4a51322997303, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:27.624507 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277939a0-a417-4549-baa6-600ca4574d53 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:27.627174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:18:27.644179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1156/2311] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:18:27 2022] PUT /identity/v3/projects/aac7de21b69441899866f7685df2e5c2/users/9a963453dec24163ada4a51322997303/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:18:27.655499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8ade9-616f-46a1-b69c-c871b258dc06 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.655699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8ade9-616f-46a1-b69c-c871b258dc06 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.655999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8ade9-616f-46a1-b69c-c871b258dc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.684074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-8ce8ade9-616f-46a1-b69c-c871b258dc06 None None] MFA Rules not processed for user `9a963453dec24163ada4a51322997303`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:18:27.685315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8ce8ade9-616f-46a1-b69c-c871b258dc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:18:27.751845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1156/2312] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:18:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:18:27.756561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277939a0-a417-4549-baa6-600ca4574d53 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:27.825170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277939a0-a417-4549-baa6-600ca4574d53 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:27.833698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbf0e1-d603-4ad9-a494-cf9bb5463095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:27.834534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbf0e1-d603-4ad9-a494-cf9bb5463095 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.834905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbf0e1-d603-4ad9-a494-cf9bb5463095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.835294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbf0e1-d603-4ad9-a494-cf9bb5463095 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.836341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbf0e1-d603-4ad9-a494-cf9bb5463095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:27.838578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbf0e1-d603-4ad9-a494-cf9bb5463095 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:27.872572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1157/2313] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:18:27 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:18:27.877376 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8ade9-616f-46a1-b69c-c871b258dc06 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:27.888534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8ade9-616f-46a1-b69c-c871b258dc06 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:27.906530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:18:27.963047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4de46b-0eb7-478a-91d6-7d38e061756c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:27.963767 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4de46b-0eb7-478a-91d6-7d38e061756c admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:27.964041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4de46b-0eb7-478a-91d6-7d38e061756c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:27.964299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4de46b-0eb7-478a-91d6-7d38e061756c admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:27.964990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4de46b-0eb7-478a-91d6-7d38e061756c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:27.966925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4de46b-0eb7-478a-91d6-7d38e061756c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:27.990266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1157/2314] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:18:27 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:18:27.994515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbf0e1-d603-4ad9-a494-cf9bb5463095 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:28.005595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbf0e1-d603-4ad9-a494-cf9bb5463095 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:28.011104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b50a3-37d4-4852-814e-96fc6dc6faf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:28.011813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b50a3-37d4-4852-814e-96fc6dc6faf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.012097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b50a3-37d4-4852-814e-96fc6dc6faf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.012373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b50a3-37d4-4852-814e-96fc6dc6faf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.013092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b50a3-37d4-4852-814e-96fc6dc6faf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:28.014710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b50a3-37d4-4852-814e-96fc6dc6faf8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:28.025880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1158/2315] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:18:28.029868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4de46b-0eb7-478a-91d6-7d38e061756c admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:28.040596 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4de46b-0eb7-478a-91d6-7d38e061756c admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:28.046419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ca8c1-7ce2-4449-a304-b1d275a46940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:28.047203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca8c1-7ce2-4449-a304-b1d275a46940 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.047522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca8c1-7ce2-4449-a304-b1d275a46940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.047799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca8c1-7ce2-4449-a304-b1d275a46940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.048528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ca8c1-7ce2-4449-a304-b1d275a46940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:28.050212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ca8c1-7ce2-4449-a304-b1d275a46940 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:28.061133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1158/2316] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:18:28.065430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b50a3-37d4-4852-814e-96fc6dc6faf8 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:28.075603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b50a3-37d4-4852-814e-96fc6dc6faf8 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:28.080612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c41a84-103c-419b-8f7d-4e82edc956e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:28.081538 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c41a84-103c-419b-8f7d-4e82edc956e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.081823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c41a84-103c-419b-8f7d-4e82edc956e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.082100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c41a84-103c-419b-8f7d-4e82edc956e1 admin admin] PATH_INFO: `/v3/projects/f7c483de7eb94f0daf588dd58b6b89d7/users/960bff9db72145bfb3908e033f5b100f/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.098112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c41a84-103c-419b-8f7d-4e82edc956e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c483de7eb94f0daf588dd58b6b89d7, user_id=960bff9db72145bfb3908e033f5b100f, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:28.100953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c41a84-103c-419b-8f7d-4e82edc956e1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:28.115790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1159/2317] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:18:28 2022] PUT /identity/v3/projects/f7c483de7eb94f0daf588dd58b6b89d7/users/960bff9db72145bfb3908e033f5b100f/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:18:28.127695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cb92d-8592-4706-ad12-f3958f1fc8eb None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.127984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cb92d-8592-4706-ad12-f3958f1fc8eb None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.128260 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cb92d-8592-4706-ad12-f3958f1fc8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.155352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-078cb92d-8592-4706-ad12-f3958f1fc8eb None None] MFA Rules not processed for user `960bff9db72145bfb3908e033f5b100f`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:18:28.156706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-078cb92d-8592-4706-ad12-f3958f1fc8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:18:28.220907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1159/2318] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:18:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:18:28.225762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c41a84-103c-419b-8f7d-4e82edc956e1 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:28.288255 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c41a84-103c-419b-8f7d-4e82edc956e1 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:28.293840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09de844-4f8d-445f-8879-72745890e5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:28.294494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09de844-4f8d-445f-8879-72745890e5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.294774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09de844-4f8d-445f-8879-72745890e5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.295045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09de844-4f8d-445f-8879-72745890e5a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.295769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09de844-4f8d-445f-8879-72745890e5a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:28.297677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09de844-4f8d-445f-8879-72745890e5a9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:28.328731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1160/2319] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:18:28 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:18:28.333206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cb92d-8592-4706-ad12-f3958f1fc8eb None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:28.344163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cb92d-8592-4706-ad12-f3958f1fc8eb None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:28.404526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55c297-dc47-4517-920e-fe4076bbc5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:28.405205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55c297-dc47-4517-920e-fe4076bbc5aa admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.405488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55c297-dc47-4517-920e-fe4076bbc5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.405762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55c297-dc47-4517-920e-fe4076bbc5aa admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.406464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55c297-dc47-4517-920e-fe4076bbc5aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:28.408543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55c297-dc47-4517-920e-fe4076bbc5aa admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:28.437251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1160/2320] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:18:28 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:18:28.441564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09de844-4f8d-445f-8879-72745890e5a9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:28.452631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09de844-4f8d-445f-8879-72745890e5a9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:28.457698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d2bbc-0afa-4ff8-93e2-ca5dd190418d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:28.458361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d2bbc-0afa-4ff8-93e2-ca5dd190418d admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.458641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d2bbc-0afa-4ff8-93e2-ca5dd190418d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.458913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d2bbc-0afa-4ff8-93e2-ca5dd190418d admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.459706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d2bbc-0afa-4ff8-93e2-ca5dd190418d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:28.461322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d2bbc-0afa-4ff8-93e2-ca5dd190418d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:28.472985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1161/2321] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:18:28.477184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55c297-dc47-4517-920e-fe4076bbc5aa admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:28.487510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55c297-dc47-4517-920e-fe4076bbc5aa admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:28.493071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1595d8-f273-45de-96f5-bfd96a63b32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:28.493952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1595d8-f273-45de-96f5-bfd96a63b32c admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.494245 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1595d8-f273-45de-96f5-bfd96a63b32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.494515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1595d8-f273-45de-96f5-bfd96a63b32c admin admin] PATH_INFO: `/v3/projects/99d7d9f2de3f4292b04c3501a063a869/users/2c08d31c15914b0eb4d034dcc6d6f483/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.510451 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1595d8-f273-45de-96f5-bfd96a63b32c admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d7d9f2de3f4292b04c3501a063a869, user_id=2c08d31c15914b0eb4d034dcc6d6f483, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:28.513145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1595d8-f273-45de-96f5-bfd96a63b32c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:28.515471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:18:28.529849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1161/2322] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:18:28 2022] PUT /identity/v3/projects/99d7d9f2de3f4292b04c3501a063a869/users/2c08d31c15914b0eb4d034dcc6d6f483/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:18:28.533978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d2bbc-0afa-4ff8-93e2-ca5dd190418d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:28.547039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d2bbc-0afa-4ff8-93e2-ca5dd190418d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:28.559947 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:18:28.606791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da450095-b94c-460f-af4d-4b4b16861c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:28.607598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da450095-b94c-460f-af4d-4b4b16861c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.607763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da450095-b94c-460f-af4d-4b4b16861c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.608043 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da450095-b94c-460f-af4d-4b4b16861c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.608770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da450095-b94c-460f-af4d-4b4b16861c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:28.610368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da450095-b94c-460f-af4d-4b4b16861c75 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:28.620758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1162/2323] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:18:28.625006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1595d8-f273-45de-96f5-bfd96a63b32c admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:18:28.635342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1595d8-f273-45de-96f5-bfd96a63b32c admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:18:28.641608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe1f23-f4d8-4ea7-a1ef-f46ef526aa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:18:28.642515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1f23-f4d8-4ea7-a1ef-f46ef526aa69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.642791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1f23-f4d8-4ea7-a1ef-f46ef526aa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.643059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1f23-f4d8-4ea7-a1ef-f46ef526aa69 admin admin] PATH_INFO: `/v3/projects/99d7d9f2de3f4292b04c3501a063a869/users/2c08d31c15914b0eb4d034dcc6d6f483/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.645230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe1f23-f4d8-4ea7-a1ef-f46ef526aa69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d7d9f2de3f4292b04c3501a063a869, user_id=2c08d31c15914b0eb4d034dcc6d6f483, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:18:28.648263 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe1f23-f4d8-4ea7-a1ef-f46ef526aa69 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:18:28.660713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1162/2324] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:18:28 2022] PUT /identity/v3/projects/99d7d9f2de3f4292b04c3501a063a869/users/2c08d31c15914b0eb4d034dcc6d6f483/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:18:28.671921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec7f1e-7608-4cda-b0e6-7f81314433d1 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:18:28.672307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec7f1e-7608-4cda-b0e6-7f81314433d1 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:18:28.672658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec7f1e-7608-4cda-b0e6-7f81314433d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:18:28.704355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-cdec7f1e-7608-4cda-b0e6-7f81314433d1 None None] MFA Rules not processed for user `2c08d31c15914b0eb4d034dcc6d6f483`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:18:28.706179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cdec7f1e-7608-4cda-b0e6-7f81314433d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:18:28.786401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1163/2325] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:18:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:18:38.656436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:INSERT=2,keystone:SELECT=7 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:18:38.783116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=55 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:19:22.251422 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe1f23-f4d8-4ea7-a1ef-f46ef526aa69 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:19:22.253373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-bafe1f23-f4d8-4ea7-a1ef-f46ef526aa69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:19:22.327807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe1f23-f4d8-4ea7-a1ef-f46ef526aa69 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:19:22.331577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae09065-28d8-4b44-acea-88601fd6ef23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:19:22.332230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae09065-28d8-4b44-acea-88601fd6ef23 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:19:22.332514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae09065-28d8-4b44-acea-88601fd6ef23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:19:22.332791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae09065-28d8-4b44-acea-88601fd6ef23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:19:22.342506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae09065-28d8-4b44-acea-88601fd6ef23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:19:22.344448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae09065-28d8-4b44-acea-88601fd6ef23 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:19:22.354824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1163/2326] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:19:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:19:32.349656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=30 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:20:23.578001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec7f1e-7608-4cda-b0e6-7f81314433d1 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:23.579674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cdec7f1e-7608-4cda-b0e6-7f81314433d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:23.593024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec7f1e-7608-4cda-b0e6-7f81314433d1 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:23.597687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a8e38-8035-47fe-8597-cc7e822123c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:23.598337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a8e38-8035-47fe-8597-cc7e822123c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:23.598614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a8e38-8035-47fe-8597-cc7e822123c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:23.598880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a8e38-8035-47fe-8597-cc7e822123c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:23.601017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-942a8e38-8035-47fe-8597-cc7e822123c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:23.690315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a8e38-8035-47fe-8597-cc7e822123c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:23.691863 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a8e38-8035-47fe-8597-cc7e822123c5 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:23.710856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1164/2327] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:20:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:20:23.869228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae09065-28d8-4b44-acea-88601fd6ef23 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:23.878861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae09065-28d8-4b44-acea-88601fd6ef23 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:23.885651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fe82a-50a3-4a2a-a2da-29325fc1ea11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:23.886631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fe82a-50a3-4a2a-a2da-29325fc1ea11 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:23.886936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fe82a-50a3-4a2a-a2da-29325fc1ea11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:23.887230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fe82a-50a3-4a2a-a2da-29325fc1ea11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:23.890486 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-507fe82a-50a3-4a2a-a2da-29325fc1ea11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:23.978083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fe82a-50a3-4a2a-a2da-29325fc1ea11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:23.979541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fe82a-50a3-4a2a-a2da-29325fc1ea11 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:23.999424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1164/2328] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:20:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:20:24.326163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a8e38-8035-47fe-8597-cc7e822123c5 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:24.328389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-942a8e38-8035-47fe-8597-cc7e822123c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:24.410969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a8e38-8035-47fe-8597-cc7e822123c5 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:24.416188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3a3f0-037a-4751-a73f-a2cb0e8da181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:24.417107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3a3f0-037a-4751-a73f-a2cb0e8da181 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:24.417504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3a3f0-037a-4751-a73f-a2cb0e8da181 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:24.417745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3a3f0-037a-4751-a73f-a2cb0e8da181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:24.419846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5fa3a3f0-037a-4751-a73f-a2cb0e8da181 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:24.486534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3a3f0-037a-4751-a73f-a2cb0e8da181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:24.488176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3a3f0-037a-4751-a73f-a2cb0e8da181 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:24.493246 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:20:24.499822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1165/2329] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:20:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:20:27.743716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fe82a-50a3-4a2a-a2da-29325fc1ea11 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:27.745165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-507fe82a-50a3-4a2a-a2da-29325fc1ea11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:27.768017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fe82a-50a3-4a2a-a2da-29325fc1ea11 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:27.773380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:27.774308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:27.774546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:27.774819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] PATH_INFO: `/v3/users/32703b5451cf48b09966a7412b747099` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:27.776046 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] RBAC: Authorizing `identity:delete_user(user_id=32703b5451cf48b09966a7412b747099)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:27.778091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:27.832807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32703b5451cf48b09966a7412b747099'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:20:27.840743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32703b5451cf48b09966a7412b747099'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:20:27.841395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32703b5451cf48b09966a7412b747099'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:20:27.849476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32703b5451cf48b09966a7412b747099'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:20:27.861110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d68b29bd-cd82-4afa-99ae-5d3985c0f23e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32703b5451cf48b09966a7412b747099'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:20:27.866949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1165/2330] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:20:27 2022] DELETE /identity/v3/users/32703b5451cf48b09966a7412b747099 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:20:28.046110 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3a3f0-037a-4751-a73f-a2cb0e8da181 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:28.058818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3a3f0-037a-4751-a73f-a2cb0e8da181 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:28.140235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:28.141038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.141317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.141593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] PATH_INFO: `/v3/projects/bdd96113d4ef451d9ffc81c03c6ea70e` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.142760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdd96113d4ef451d9ffc81c03c6ea70e)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:28.145065 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:28.193800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdd96113d4ef451d9ffc81c03c6ea70e', 'request_id': 'req-e15b7b04-e025-41a6-aeec-23a6b3f31d17'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:20:28.200649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdd96113d4ef451d9ffc81c03c6ea70e', 'request_id': 'req-e15b7b04-e025-41a6-aeec-23a6b3f31d17'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:20:28.206761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdd96113d4ef451d9ffc81c03c6ea70e', 'request_id': 'req-e15b7b04-e025-41a6-aeec-23a6b3f31d17'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:20:28.234757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] The token cache is being invalidate because project bdd96113d4ef451d9ffc81c03c6ea70e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:20:28.234946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:20:28.235599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e15b7b04-e025-41a6-aeec-23a6b3f31d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:20:28.237185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1166/2331] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:20:28 2022] DELETE /identity/v3/projects/bdd96113d4ef451d9ffc81c03c6ea70e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:20:28.243156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0562746-f696-401d-b768-371220f94c8a None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.243578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0562746-f696-401d-b768-371220f94c8a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.243856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0562746-f696-401d-b768-371220f94c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.262682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d0562746-f696-401d-b768-371220f94c8a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:20:28.263953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d0562746-f696-401d-b768-371220f94c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:28.305384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:20:28.328564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1166/2332] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:20:28.336621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71a58b-0f33-4d8b-8815-e09fc317989b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.336906 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71a58b-0f33-4d8b-8815-e09fc317989b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.337177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71a58b-0f33-4d8b-8815-e09fc317989b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.355769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ba71a58b-0f33-4d8b-8815-e09fc317989b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:20:28.356981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ba71a58b-0f33-4d8b-8815-e09fc317989b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:28.366505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1167/2333] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:20:28.370878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0562746-f696-401d-b768-371220f94c8a None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:28.381377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0562746-f696-401d-b768-371220f94c8a None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:28.387360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11492053-88cb-4a9b-aa3d-147ae1576428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:28.388018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11492053-88cb-4a9b-aa3d-147ae1576428 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.388296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11492053-88cb-4a9b-aa3d-147ae1576428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.388571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11492053-88cb-4a9b-aa3d-147ae1576428 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.389300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11492053-88cb-4a9b-aa3d-147ae1576428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:20:28.389609 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11492053-88cb-4a9b-aa3d-147ae1576428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:28.391225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11492053-88cb-4a9b-aa3d-147ae1576428 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:28.403857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1167/2334] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:20:28.408256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71a58b-0f33-4d8b-8815-e09fc317989b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:28.418033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71a58b-0f33-4d8b-8815-e09fc317989b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:28.423953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0495ade-8aea-496d-957a-cee9798fdeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:28.424640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0495ade-8aea-496d-957a-cee9798fdeaf admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.424925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0495ade-8aea-496d-957a-cee9798fdeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.425201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0495ade-8aea-496d-957a-cee9798fdeaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.425898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0495ade-8aea-496d-957a-cee9798fdeaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:28.427865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0495ade-8aea-496d-957a-cee9798fdeaf admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:28.458069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1168/2335] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:20:28 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:20:28.462329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11492053-88cb-4a9b-aa3d-147ae1576428 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:28.472831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11492053-88cb-4a9b-aa3d-147ae1576428 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:28.534020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9585b6-d7fc-415e-86ff-345ce468233d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:28.534727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9585b6-d7fc-415e-86ff-345ce468233d admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.535009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9585b6-d7fc-415e-86ff-345ce468233d admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.535323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9585b6-d7fc-415e-86ff-345ce468233d admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.536031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9585b6-d7fc-415e-86ff-345ce468233d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:28.537971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9585b6-d7fc-415e-86ff-345ce468233d admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:28.560899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1168/2336] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:20:28 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:20:28.565159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0495ade-8aea-496d-957a-cee9798fdeaf admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:28.575889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0495ade-8aea-496d-957a-cee9798fdeaf admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:28.580694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa894659-8695-44be-b0a8-a16c63c87023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:28.581355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894659-8695-44be-b0a8-a16c63c87023 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.581628 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894659-8695-44be-b0a8-a16c63c87023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.582175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894659-8695-44be-b0a8-a16c63c87023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.582913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa894659-8695-44be-b0a8-a16c63c87023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:28.584576 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa894659-8695-44be-b0a8-a16c63c87023 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:28.594658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1169/2337] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:20:28.598655 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9585b6-d7fc-415e-86ff-345ce468233d admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:28.608967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9585b6-d7fc-415e-86ff-345ce468233d admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:28.614121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c68ffc-9608-4553-90b2-496d216a834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:28.614765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c68ffc-9608-4553-90b2-496d216a834a admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.615035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c68ffc-9608-4553-90b2-496d216a834a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.615338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c68ffc-9608-4553-90b2-496d216a834a admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.616065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c68ffc-9608-4553-90b2-496d216a834a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:28.617679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c68ffc-9608-4553-90b2-496d216a834a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:28.628166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1169/2338] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:20:28.634257 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa894659-8695-44be-b0a8-a16c63c87023 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:28.646014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa894659-8695-44be-b0a8-a16c63c87023 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:28.651776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e4b5d-6980-4458-9f2d-fc126a554fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:28.652685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e4b5d-6980-4458-9f2d-fc126a554fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.652961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e4b5d-6980-4458-9f2d-fc126a554fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.653233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e4b5d-6980-4458-9f2d-fc126a554fa9 admin admin] PATH_INFO: `/v3/projects/79f8be4bddfe4c0c8ebf013f7c183d9e/users/d3b658858f7642b1a3fbf2b8aef3b538/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.669713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e4b5d-6980-4458-9f2d-fc126a554fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f8be4bddfe4c0c8ebf013f7c183d9e, user_id=d3b658858f7642b1a3fbf2b8aef3b538, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:28.672409 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e4b5d-6980-4458-9f2d-fc126a554fa9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:28.682249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:20:28.688528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1170/2339] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:20:28 2022] PUT /identity/v3/projects/79f8be4bddfe4c0c8ebf013f7c183d9e/users/d3b658858f7642b1a3fbf2b8aef3b538/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:20:28.699430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94561814-b37d-4922-87ca-ed1a608d2352 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.699704 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94561814-b37d-4922-87ca-ed1a608d2352 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.699985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94561814-b37d-4922-87ca-ed1a608d2352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.727821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-94561814-b37d-4922-87ca-ed1a608d2352 None None] MFA Rules not processed for user `d3b658858f7642b1a3fbf2b8aef3b538`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:20:28.729019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-94561814-b37d-4922-87ca-ed1a608d2352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:28.812887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1170/2340] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:20:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:20:28.817405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e4b5d-6980-4458-9f2d-fc126a554fa9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:28.879348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e4b5d-6980-4458-9f2d-fc126a554fa9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:28.884468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbee5cd-7072-43ad-ae7e-2bbc41ce6312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:28.885243 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbee5cd-7072-43ad-ae7e-2bbc41ce6312 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.885592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbee5cd-7072-43ad-ae7e-2bbc41ce6312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.885939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbee5cd-7072-43ad-ae7e-2bbc41ce6312 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.886644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbee5cd-7072-43ad-ae7e-2bbc41ce6312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:28.888623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbee5cd-7072-43ad-ae7e-2bbc41ce6312 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:28.921369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1171/2341] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:20:28 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:20:28.925134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94561814-b37d-4922-87ca-ed1a608d2352 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:28.936856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94561814-b37d-4922-87ca-ed1a608d2352 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:28.970035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:20:28.996684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4b882-5634-4940-b6f6-0b1b2631b524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:28.997362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4b882-5634-4940-b6f6-0b1b2631b524 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:28.997675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4b882-5634-4940-b6f6-0b1b2631b524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:28.997958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4b882-5634-4940-b6f6-0b1b2631b524 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:28.998659 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4b882-5634-4940-b6f6-0b1b2631b524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:29.000632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4b882-5634-4940-b6f6-0b1b2631b524 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:29.022579 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1171/2342] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:20:28 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:20:29.026734 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbee5cd-7072-43ad-ae7e-2bbc41ce6312 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:29.037435 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbee5cd-7072-43ad-ae7e-2bbc41ce6312 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:29.042582 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7197a-6b74-49a7-a9c4-519b94ae0dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:29.043236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7197a-6b74-49a7-a9c4-519b94ae0dda admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:29.043565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7197a-6b74-49a7-a9c4-519b94ae0dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:29.043840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7197a-6b74-49a7-a9c4-519b94ae0dda admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:29.044574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7197a-6b74-49a7-a9c4-519b94ae0dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:29.046181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7197a-6b74-49a7-a9c4-519b94ae0dda admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:29.057392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1172/2343] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:20:29.061617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4b882-5634-4940-b6f6-0b1b2631b524 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:29.071873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4b882-5634-4940-b6f6-0b1b2631b524 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:29.076756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b07c1-cff6-4782-b8ed-5fc1cd70609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:29.077639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b07c1-cff6-4782-b8ed-5fc1cd70609a admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:29.077919 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b07c1-cff6-4782-b8ed-5fc1cd70609a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:29.078192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b07c1-cff6-4782-b8ed-5fc1cd70609a admin admin] PATH_INFO: `/v3/projects/16a5cde15932411d895e4d87545e466e/users/72feedf18f884756a04dcaa1271e0656/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:29.093258 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b07c1-cff6-4782-b8ed-5fc1cd70609a admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a5cde15932411d895e4d87545e466e, user_id=72feedf18f884756a04dcaa1271e0656, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:29.095981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b07c1-cff6-4782-b8ed-5fc1cd70609a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:29.111151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1172/2344] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:20:29 2022] PUT /identity/v3/projects/16a5cde15932411d895e4d87545e466e/users/72feedf18f884756a04dcaa1271e0656/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:20:29.115266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7197a-6b74-49a7-a9c4-519b94ae0dda admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:29.127693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7197a-6b74-49a7-a9c4-519b94ae0dda admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:29.185524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b76d99-b5b9-48d3-94d5-b48c02565186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:29.186219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b76d99-b5b9-48d3-94d5-b48c02565186 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:29.186503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b76d99-b5b9-48d3-94d5-b48c02565186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:29.186780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b76d99-b5b9-48d3-94d5-b48c02565186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:29.187541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b76d99-b5b9-48d3-94d5-b48c02565186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:29.189157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b76d99-b5b9-48d3-94d5-b48c02565186 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:29.199449 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1173/2345] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:20:29.204150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b07c1-cff6-4782-b8ed-5fc1cd70609a admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:29.218793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b07c1-cff6-4782-b8ed-5fc1cd70609a admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:29.224700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9eb3c6-1336-475d-a6e4-bc577adc298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:29.225587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9eb3c6-1336-475d-a6e4-bc577adc298f admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:29.225857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9eb3c6-1336-475d-a6e4-bc577adc298f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:29.226111 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9eb3c6-1336-475d-a6e4-bc577adc298f admin admin] PATH_INFO: `/v3/projects/16a5cde15932411d895e4d87545e466e/users/72feedf18f884756a04dcaa1271e0656/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:29.228345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9eb3c6-1336-475d-a6e4-bc577adc298f admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a5cde15932411d895e4d87545e466e, user_id=72feedf18f884756a04dcaa1271e0656, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:29.231119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9eb3c6-1336-475d-a6e4-bc577adc298f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:29.241668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1173/2346] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:20:29 2022] PUT /identity/v3/projects/16a5cde15932411d895e4d87545e466e/users/72feedf18f884756a04dcaa1271e0656/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:20:29.251897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3558662-23cd-4056-b3f9-2be3cece8504 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:29.252195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3558662-23cd-4056-b3f9-2be3cece8504 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:29.252474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3558662-23cd-4056-b3f9-2be3cece8504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:29.279423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a3558662-23cd-4056-b3f9-2be3cece8504 None None] MFA Rules not processed for user `72feedf18f884756a04dcaa1271e0656`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:20:29.280635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a3558662-23cd-4056-b3f9-2be3cece8504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:29.345764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:20:29.356721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1174/2347] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:20:29 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:20:31.827930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9eb3c6-1336-475d-a6e4-bc577adc298f admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:20:31.829894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0f9eb3c6-1336-475d-a6e4-bc577adc298f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:31.892238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9eb3c6-1336-475d-a6e4-bc577adc298f admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:20:31.897371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96afedb-f205-4980-a8eb-2ade13c02409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:20:31.898052 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96afedb-f205-4980-a8eb-2ade13c02409 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:20:31.898335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96afedb-f205-4980-a8eb-2ade13c02409 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:20:31.898611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96afedb-f205-4980-a8eb-2ade13c02409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:20:31.900735 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d96afedb-f205-4980-a8eb-2ade13c02409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:20:31.912496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96afedb-f205-4980-a8eb-2ade13c02409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:20:31.913933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96afedb-f205-4980-a8eb-2ade13c02409 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:20:31.922236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1174/2348] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:20:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:20:39.357803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:20:41.918865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=53,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:21:22.549087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3558662-23cd-4056-b3f9-2be3cece8504 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:21:22.549087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a3558662-23cd-4056-b3f9-2be3cece8504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:21:22.607111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3558662-23cd-4056-b3f9-2be3cece8504 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:21:22.610931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac44824-de21-423e-9c81-b6afb3f3a794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:21:22.611588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac44824-de21-423e-9c81-b6afb3f3a794 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:21:22.611870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac44824-de21-423e-9c81-b6afb3f3a794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:21:22.612172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac44824-de21-423e-9c81-b6afb3f3a794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:21:22.614049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0ac44824-de21-423e-9c81-b6afb3f3a794 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:21:22.695461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac44824-de21-423e-9c81-b6afb3f3a794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:21:22.696840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac44824-de21-423e-9c81-b6afb3f3a794 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:21:22.706215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1175/2349] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:21:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:21:22.794607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96afedb-f205-4980-a8eb-2ade13c02409 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:21:22.804487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96afedb-f205-4980-a8eb-2ade13c02409 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:21:22.808772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee027-fd03-4495-8a4e-611ae05599b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:21:22.809408 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee027-fd03-4495-8a4e-611ae05599b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:21:22.809694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee027-fd03-4495-8a4e-611ae05599b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:21:22.809968 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee027-fd03-4495-8a4e-611ae05599b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:21:22.820308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ee027-fd03-4495-8a4e-611ae05599b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:21:22.822007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ee027-fd03-4495-8a4e-611ae05599b6 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:21:22.832734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1175/2350] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:21:22 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:21:23.322889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac44824-de21-423e-9c81-b6afb3f3a794 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:21:23.332567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac44824-de21-423e-9c81-b6afb3f3a794 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:21:23.336909 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa78ac-c9ab-4576-8338-d725b528292f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:21:23.337591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa78ac-c9ab-4576-8338-d725b528292f service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:21:23.337876 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa78ac-c9ab-4576-8338-d725b528292f service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:21:23.338151 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa78ac-c9ab-4576-8338-d725b528292f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:21:23.340264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-61aa78ac-c9ab-4576-8338-d725b528292f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:21:23.399664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa78ac-c9ab-4576-8338-d725b528292f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:21:23.401162 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa78ac-c9ab-4576-8338-d725b528292f service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:21:23.410795 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1176/2351] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:21:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:21:23.580598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee027-fd03-4495-8a4e-611ae05599b6 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:21:23.582303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-173ee027-fd03-4495-8a4e-611ae05599b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:21:23.706686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee027-fd03-4495-8a4e-611ae05599b6 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:21:23.716324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ecc34-0280-4173-adbf-dbd1e02122f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:21:23.717122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ecc34-0280-4173-adbf-dbd1e02122f6 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:21:23.717556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ecc34-0280-4173-adbf-dbd1e02122f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:21:23.718005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ecc34-0280-4173-adbf-dbd1e02122f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:21:23.720020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6a6ecc34-0280-4173-adbf-dbd1e02122f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:21:23.733681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ecc34-0280-4173-adbf-dbd1e02122f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:21:23.735297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ecc34-0280-4173-adbf-dbd1e02122f6 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:21:23.746790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1176/2352] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:21:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:21:33.406680 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=81 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:21:33.742645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=52 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:23.065210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa78ac-c9ab-4576-8338-d725b528292f service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:23.067494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-61aa78ac-c9ab-4576-8338-d725b528292f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:23.161119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa78ac-c9ab-4576-8338-d725b528292f service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:23.166338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279aea0-6bad-4d71-9283-95f73edf9162 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:23.167012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279aea0-6bad-4d71-9283-95f73edf9162 service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:23.167327 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279aea0-6bad-4d71-9283-95f73edf9162 service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:23.167609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279aea0-6bad-4d71-9283-95f73edf9162 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:23.169658 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-1279aea0-6bad-4d71-9283-95f73edf9162 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:23.181735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279aea0-6bad-4d71-9283-95f73edf9162 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:23.183400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279aea0-6bad-4d71-9283-95f73edf9162 service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:23.197234 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1177/2353] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:22:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:22:24.384488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ecc34-0280-4173-adbf-dbd1e02122f6 service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:24.385917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6a6ecc34-0280-4173-adbf-dbd1e02122f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:24.398028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ecc34-0280-4173-adbf-dbd1e02122f6 service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:24.401789 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43227710-4ec4-4e4a-bc8b-bb166b0c7530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:24.402427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43227710-4ec4-4e4a-bc8b-bb166b0c7530 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:24.402708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43227710-4ec4-4e4a-bc8b-bb166b0c7530 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:24.402981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43227710-4ec4-4e4a-bc8b-bb166b0c7530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:24.404819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-43227710-4ec4-4e4a-bc8b-bb166b0c7530 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:24.478681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43227710-4ec4-4e4a-bc8b-bb166b0c7530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:24.480222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43227710-4ec4-4e4a-bc8b-bb166b0c7530 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:24.490297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1177/2354] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:22:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:22:24.853158 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279aea0-6bad-4d71-9283-95f73edf9162 service placement] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:24.916244 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279aea0-6bad-4d71-9283-95f73edf9162 service placement] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:24.921065 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:24.921953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:24.922231 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:24.922504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] PATH_INFO: `/v3/users/d3b658858f7642b1a3fbf2b8aef3b538` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:24.923761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b658858f7642b1a3fbf2b8aef3b538)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:24.925707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:24.987983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b658858f7642b1a3fbf2b8aef3b538'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:24.995633 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b658858f7642b1a3fbf2b8aef3b538'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.007529 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b658858f7642b1a3fbf2b8aef3b538'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.008250 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b658858f7642b1a3fbf2b8aef3b538'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.008954 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b658858f7642b1a3fbf2b8aef3b538'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.022925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:25.028898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1178/2355] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:22:24 2022] DELETE /identity/v3/users/d3b658858f7642b1a3fbf2b8aef3b538 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:25.033701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c01bc-59d7-4b4f-adab-eab6d7d0ccca None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:25.033985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c01bc-59d7-4b4f-adab-eab6d7d0ccca None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:25.034258 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c01bc-59d7-4b4f-adab-eab6d7d0ccca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:25.051836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-694c01bc-59d7-4b4f-adab-eab6d7d0ccca None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:22:25.053055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-694c01bc-59d7-4b4f-adab-eab6d7d0ccca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:25.116329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1178/2356] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:22:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:22:25.127868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:25.129579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:25.189142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758b01a-254f-4f8b-8c8d-3bdbd0a0cf23 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:25.193553 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9746c2-7651-4570-95f9-aef563dd6cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:25.194223 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9746c2-7651-4570-95f9-aef563dd6cac service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:25.194485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9746c2-7651-4570-95f9-aef563dd6cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:25.194766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9746c2-7651-4570-95f9-aef563dd6cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:25.204731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9746c2-7651-4570-95f9-aef563dd6cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:25.206292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9746c2-7651-4570-95f9-aef563dd6cac service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:25.216063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1179/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:22:25.501940 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c01bc-59d7-4b4f-adab-eab6d7d0ccca None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:25.503493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-694c01bc-59d7-4b4f-adab-eab6d7d0ccca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:25.517879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c01bc-59d7-4b4f-adab-eab6d7d0ccca None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:25.523276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:25.524148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:25.524427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:25.524703 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] PATH_INFO: `/v3/projects/79f8be4bddfe4c0c8ebf013f7c183d9e` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:25.525948 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f8be4bddfe4c0c8ebf013f7c183d9e)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:25.527978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:25.580113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f8be4bddfe4c0c8ebf013f7c183d9e', 'request_id': 'req-ce81609b-9a13-4502-bd16-89d09db86d9d'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.586409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f8be4bddfe4c0c8ebf013f7c183d9e', 'request_id': 'req-ce81609b-9a13-4502-bd16-89d09db86d9d'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.592122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f8be4bddfe4c0c8ebf013f7c183d9e', 'request_id': 'req-ce81609b-9a13-4502-bd16-89d09db86d9d'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.607906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] The token cache is being invalidate because project 79f8be4bddfe4c0c8ebf013f7c183d9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:22:25.608140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.609108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.613515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1179/2358] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:22:25 2022] DELETE /identity/v3/projects/79f8be4bddfe4c0c8ebf013f7c183d9e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:25.614572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9746c2-7651-4570-95f9-aef563dd6cac service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:25.630501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9746c2-7651-4570-95f9-aef563dd6cac service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:25.702720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:25.703654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:25.703933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:25.704203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] PATH_INFO: `/v3/users/72feedf18f884756a04dcaa1271e0656` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:25.705380 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72feedf18f884756a04dcaa1271e0656)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:25.707322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:25.761280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72feedf18f884756a04dcaa1271e0656'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.769029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72feedf18f884756a04dcaa1271e0656'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.778815 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72feedf18f884756a04dcaa1271e0656'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.779538 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72feedf18f884756a04dcaa1271e0656'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.780152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a0f9614c-8684-4aa9-b9fe-dea839e002f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72feedf18f884756a04dcaa1271e0656'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:25.797925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1180/2359] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:22:25 2022] DELETE /identity/v3/users/72feedf18f884756a04dcaa1271e0656 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:26.082483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:26.083972 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:26.098070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81609b-9a13-4502-bd16-89d09db86d9d admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:26.106863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:26.158963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:26.159772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.160075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.160354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] PATH_INFO: `/v3/projects/16a5cde15932411d895e4d87545e466e` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.161599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16a5cde15932411d895e4d87545e466e)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:26.163569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:26.211126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16a5cde15932411d895e4d87545e466e', 'request_id': 'req-662930dd-1fd1-4f04-a1ec-1d3493526e11'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:26.217137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16a5cde15932411d895e4d87545e466e', 'request_id': 'req-662930dd-1fd1-4f04-a1ec-1d3493526e11'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:26.223772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16a5cde15932411d895e4d87545e466e', 'request_id': 'req-662930dd-1fd1-4f04-a1ec-1d3493526e11'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:26.241612 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] The token cache is being invalidate because project 16a5cde15932411d895e4d87545e466e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:22:26.241989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:26.242875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-662930dd-1fd1-4f04-a1ec-1d3493526e11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:26.245045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1180/2360] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:22:26 2022] DELETE /identity/v3/projects/16a5cde15932411d895e4d87545e466e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:26.250294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138df83-765b-4d96-a7fe-106b0f3ccbdb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.250802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138df83-765b-4d96-a7fe-106b0f3ccbdb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.250802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138df83-765b-4d96-a7fe-106b0f3ccbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.281092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2138df83-765b-4d96-a7fe-106b0f3ccbdb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:22:26.282312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2138df83-765b-4d96-a7fe-106b0f3ccbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:26.287866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:26.351707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1181/2361] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:22:26.361518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b84a99-40b1-45a3-8d01-8bbbe460f1d3 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.361811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b84a99-40b1-45a3-8d01-8bbbe460f1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.362085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b84a99-40b1-45a3-8d01-8bbbe460f1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.383271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-f2b84a99-40b1-45a3-8d01-8bbbe460f1d3 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:22:26.384498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f2b84a99-40b1-45a3-8d01-8bbbe460f1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:26.399890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1181/2362] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:22:26.404478 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138df83-765b-4d96-a7fe-106b0f3ccbdb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:26.414953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138df83-765b-4d96-a7fe-106b0f3ccbdb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:26.420638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27bfd-fc96-4f99-802c-3c49304c68c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:26.421281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27bfd-fc96-4f99-802c-3c49304c68c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.421589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27bfd-fc96-4f99-802c-3c49304c68c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.421872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27bfd-fc96-4f99-802c-3c49304c68c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.422609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e27bfd-fc96-4f99-802c-3c49304c68c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:22:26.422921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e27bfd-fc96-4f99-802c-3c49304c68c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:26.424574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e27bfd-fc96-4f99-802c-3c49304c68c1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:26.436630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1182/2363] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:22:26.440956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b84a99-40b1-45a3-8d01-8bbbe460f1d3 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:26.450486 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b84a99-40b1-45a3-8d01-8bbbe460f1d3 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:26.455715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52bb6b-4da8-4cad-94be-6f97f8ee6f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:26.456369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52bb6b-4da8-4cad-94be-6f97f8ee6f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.456646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52bb6b-4da8-4cad-94be-6f97f8ee6f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.456924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52bb6b-4da8-4cad-94be-6f97f8ee6f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.457628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52bb6b-4da8-4cad-94be-6f97f8ee6f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:26.459570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52bb6b-4da8-4cad-94be-6f97f8ee6f75 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:26.488958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1182/2364] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:22:26 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:22:26.493361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27bfd-fc96-4f99-802c-3c49304c68c1 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:26.503818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27bfd-fc96-4f99-802c-3c49304c68c1 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:26.562075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd76c97-3bd8-4d17-9ec8-46351b17a2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:26.562768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd76c97-3bd8-4d17-9ec8-46351b17a2b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.563052 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd76c97-3bd8-4d17-9ec8-46351b17a2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.563358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd76c97-3bd8-4d17-9ec8-46351b17a2b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.564064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd76c97-3bd8-4d17-9ec8-46351b17a2b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:26.566040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd76c97-3bd8-4d17-9ec8-46351b17a2b0 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:26.588779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1183/2365] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:22:26 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:22:26.593072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52bb6b-4da8-4cad-94be-6f97f8ee6f75 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:26.606598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52bb6b-4da8-4cad-94be-6f97f8ee6f75 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:26.612382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acffb0b-0dd1-48b9-bc2a-bee03afa6db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:26.613044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acffb0b-0dd1-48b9-bc2a-bee03afa6db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.613331 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acffb0b-0dd1-48b9-bc2a-bee03afa6db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.613600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acffb0b-0dd1-48b9-bc2a-bee03afa6db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.614357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acffb0b-0dd1-48b9-bc2a-bee03afa6db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:26.616048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acffb0b-0dd1-48b9-bc2a-bee03afa6db3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:26.627173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1183/2366] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:22:26.632173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd76c97-3bd8-4d17-9ec8-46351b17a2b0 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:26.643729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd76c97-3bd8-4d17-9ec8-46351b17a2b0 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:26.648836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f5c08-1c30-4620-819c-364104fb9f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:26.649518 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f5c08-1c30-4620-819c-364104fb9f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.649807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f5c08-1c30-4620-819c-364104fb9f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.650082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f5c08-1c30-4620-819c-364104fb9f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.650850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f5c08-1c30-4620-819c-364104fb9f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:26.652490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f5c08-1c30-4620-819c-364104fb9f11 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:26.663798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1184/2367] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:22:26.667971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acffb0b-0dd1-48b9-bc2a-bee03afa6db3 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:26.680438 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acffb0b-0dd1-48b9-bc2a-bee03afa6db3 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:26.686148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93239eab-29a7-4887-a7f3-608edef0f1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:26.687067 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93239eab-29a7-4887-a7f3-608edef0f1db admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.687830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93239eab-29a7-4887-a7f3-608edef0f1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.687977 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93239eab-29a7-4887-a7f3-608edef0f1db admin admin] PATH_INFO: `/v3/projects/40cf964699b64181908e8db3fbaae655/users/daa09c3db4ad4697b758ec77f1887d17/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.704866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93239eab-29a7-4887-a7f3-608edef0f1db admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cf964699b64181908e8db3fbaae655, user_id=daa09c3db4ad4697b758ec77f1887d17, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:26.707562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93239eab-29a7-4887-a7f3-608edef0f1db admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:26.722575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1184/2368] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:22:26 2022] PUT /identity/v3/projects/40cf964699b64181908e8db3fbaae655/users/daa09c3db4ad4697b758ec77f1887d17/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:26.733982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8380-b9bb-485a-9180-5aa2d4e4fa12 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.734400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8380-b9bb-485a-9180-5aa2d4e4fa12 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.734680 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8380-b9bb-485a-9180-5aa2d4e4fa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.762704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-b78a8380-b9bb-485a-9180-5aa2d4e4fa12 None None] MFA Rules not processed for user `daa09c3db4ad4697b758ec77f1887d17`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:22:26.763957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b78a8380-b9bb-485a-9180-5aa2d4e4fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:26.839185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1185/2369] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:22:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:22:26.843801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93239eab-29a7-4887-a7f3-608edef0f1db admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:26.883886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:26.908241 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93239eab-29a7-4887-a7f3-608edef0f1db admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:26.913139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae7c29-eb36-40e4-9e68-bface721bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:26.913792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae7c29-eb36-40e4-9e68-bface721bbdf admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:26.914077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae7c29-eb36-40e4-9e68-bface721bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:26.914352 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae7c29-eb36-40e4-9e68-bface721bbdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:26.915056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae7c29-eb36-40e4-9e68-bface721bbdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:26.917023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae7c29-eb36-40e4-9e68-bface721bbdf admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:26.943910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1185/2370] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:22:26 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:22:26.948062 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a8380-b9bb-485a-9180-5aa2d4e4fa12 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:26.958828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a8380-b9bb-485a-9180-5aa2d4e4fa12 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:26.962776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:27.024060 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1beb3b-51db-4031-a2d2-858981e660d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:27.024784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1beb3b-51db-4031-a2d2-858981e660d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:27.025067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1beb3b-51db-4031-a2d2-858981e660d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:27.025338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1beb3b-51db-4031-a2d2-858981e660d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:27.026033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1beb3b-51db-4031-a2d2-858981e660d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:27.028448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1beb3b-51db-4031-a2d2-858981e660d5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:27.052593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1186/2371] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:22:26 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:22:27.056925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae7c29-eb36-40e4-9e68-bface721bbdf admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:27.068397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae7c29-eb36-40e4-9e68-bface721bbdf admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:27.074565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f70abf-6651-411c-8b23-8d89ddb8c94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:27.075226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f70abf-6651-411c-8b23-8d89ddb8c94b admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:27.075544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f70abf-6651-411c-8b23-8d89ddb8c94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:27.075810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f70abf-6651-411c-8b23-8d89ddb8c94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:27.076543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f70abf-6651-411c-8b23-8d89ddb8c94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:27.078151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f70abf-6651-411c-8b23-8d89ddb8c94b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:27.089673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1186/2372] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:22:27.094325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1beb3b-51db-4031-a2d2-858981e660d5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:27.104830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1beb3b-51db-4031-a2d2-858981e660d5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:27.110236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6deb0-2e31-42ce-92e2-bca7385fc4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:27.111132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6deb0-2e31-42ce-92e2-bca7385fc4b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:27.111448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6deb0-2e31-42ce-92e2-bca7385fc4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:27.111741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6deb0-2e31-42ce-92e2-bca7385fc4b5 admin admin] PATH_INFO: `/v3/projects/8c05ccf515864b3395e7b2b367036796/users/3bd5d96ebc5042748373951a9d4120e6/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:27.126753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6deb0-2e31-42ce-92e2-bca7385fc4b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c05ccf515864b3395e7b2b367036796, user_id=3bd5d96ebc5042748373951a9d4120e6, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:27.129454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6deb0-2e31-42ce-92e2-bca7385fc4b5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:27.145438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1187/2373] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:22:27 2022] PUT /identity/v3/projects/8c05ccf515864b3395e7b2b367036796/users/3bd5d96ebc5042748373951a9d4120e6/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:27.149406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f70abf-6651-411c-8b23-8d89ddb8c94b admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:27.159495 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f70abf-6651-411c-8b23-8d89ddb8c94b admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:27.218047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa08247-322d-4454-ad1e-b3f37e5884f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:27.218654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08247-322d-4454-ad1e-b3f37e5884f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:27.218949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08247-322d-4454-ad1e-b3f37e5884f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:27.219202 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08247-322d-4454-ad1e-b3f37e5884f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:27.219956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa08247-322d-4454-ad1e-b3f37e5884f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:27.222032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa08247-322d-4454-ad1e-b3f37e5884f5 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:27.232397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1187/2374] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:22:27.238488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6deb0-2e31-42ce-92e2-bca7385fc4b5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:27.249567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6deb0-2e31-42ce-92e2-bca7385fc4b5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:27.255092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a375ea-101d-47db-a75b-938813914b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:27.256028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a375ea-101d-47db-a75b-938813914b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:27.256307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a375ea-101d-47db-a75b-938813914b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:27.256577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a375ea-101d-47db-a75b-938813914b37 admin admin] PATH_INFO: `/v3/projects/8c05ccf515864b3395e7b2b367036796/users/3bd5d96ebc5042748373951a9d4120e6/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:27.258770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a375ea-101d-47db-a75b-938813914b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c05ccf515864b3395e7b2b367036796, user_id=3bd5d96ebc5042748373951a9d4120e6, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:27.261502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a375ea-101d-47db-a75b-938813914b37 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:27.273110 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1188/2375] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:22:27 2022] PUT /identity/v3/projects/8c05ccf515864b3395e7b2b367036796/users/3bd5d96ebc5042748373951a9d4120e6/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:27.284154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b659f-293b-462a-b978-1ae77f107dfe None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:27.284504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b659f-293b-462a-b978-1ae77f107dfe None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:27.284849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b659f-293b-462a-b978-1ae77f107dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:27.315453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a75b659f-293b-462a-b978-1ae77f107dfe None None] MFA Rules not processed for user `3bd5d96ebc5042748373951a9d4120e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:22:27.316664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a75b659f-293b-462a-b978-1ae77f107dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:27.399334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1188/2376] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:22:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:22:27.412020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a375ea-101d-47db-a75b-938813914b37 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:27.414162 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-62a375ea-101d-47db-a75b-938813914b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:27.475994 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a375ea-101d-47db-a75b-938813914b37 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:27.480019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318867a2-f857-4a83-b64b-3ba795c96613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:27.480666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318867a2-f857-4a83-b64b-3ba795c96613 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:27.480948 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318867a2-f857-4a83-b64b-3ba795c96613 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:27.481221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318867a2-f857-4a83-b64b-3ba795c96613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:27.491023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318867a2-f857-4a83-b64b-3ba795c96613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:27.492491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318867a2-f857-4a83-b64b-3ba795c96613 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:27.502206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1189/2377] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:22:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:22:27.530611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b659f-293b-462a-b978-1ae77f107dfe None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:27.564163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:27.592935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b659f-293b-462a-b978-1ae77f107dfe None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:27.597608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:27.598490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:27.598780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:27.599035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] PATH_INFO: `/v3/users/daa09c3db4ad4697b758ec77f1887d17` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:27.600266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daa09c3db4ad4697b758ec77f1887d17)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:27.602197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:27.652245 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daa09c3db4ad4697b758ec77f1887d17'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:27.659145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daa09c3db4ad4697b758ec77f1887d17'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:27.659875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daa09c3db4ad4697b758ec77f1887d17'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:27.668636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daa09c3db4ad4697b758ec77f1887d17'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:27.680563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daa09c3db4ad4697b758ec77f1887d17'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:27.686852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1189/2378] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:22:27 2022] DELETE /identity/v3/users/daa09c3db4ad4697b758ec77f1887d17 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:27.691401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf54f8-f8bd-4449-bd10-b7753262a198 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:27.691690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf54f8-f8bd-4449-bd10-b7753262a198 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:27.691969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf54f8-f8bd-4449-bd10-b7753262a198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:27.713504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-8fdf54f8-f8bd-4449-bd10-b7753262a198 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:22:27.714712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8fdf54f8-f8bd-4449-bd10-b7753262a198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:27.778344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1190/2379] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:22:27.788058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:27.789728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:27.851911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e77e06-a88d-4e03-ba8d-abcdd47978a0 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:27.855540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812fec5-fdc1-4584-b27f-65e66cef2da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:27.856221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812fec5-fdc1-4584-b27f-65e66cef2da4 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:27.856465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812fec5-fdc1-4584-b27f-65e66cef2da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:27.856732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812fec5-fdc1-4584-b27f-65e66cef2da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:27.865967 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812fec5-fdc1-4584-b27f-65e66cef2da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:27.867473 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812fec5-fdc1-4584-b27f-65e66cef2da4 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:27.877383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1190/2380] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:22:28.146336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf54f8-f8bd-4449-bd10-b7753262a198 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:28.147809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8fdf54f8-f8bd-4449-bd10-b7753262a198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:28.154743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:28.160475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf54f8-f8bd-4449-bd10-b7753262a198 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:28.166321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:28.167495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:28.167748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:28.168043 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] PATH_INFO: `/v3/projects/40cf964699b64181908e8db3fbaae655` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:28.169275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cf964699b64181908e8db3fbaae655)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:28.171282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:28.223266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cf964699b64181908e8db3fbaae655', 'request_id': 'req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.229899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cf964699b64181908e8db3fbaae655', 'request_id': 'req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.236209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cf964699b64181908e8db3fbaae655', 'request_id': 'req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.251680 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] The token cache is being invalidate because project 40cf964699b64181908e8db3fbaae655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:22:28.251973 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.252654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.254291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1191/2381] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:22:28 2022] DELETE /identity/v3/projects/40cf964699b64181908e8db3fbaae655 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:28.258576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812fec5-fdc1-4584-b27f-65e66cef2da4 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:28.268853 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812fec5-fdc1-4584-b27f-65e66cef2da4 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:28.320754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:28.328695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:28.329588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:28.329876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:28.330157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] PATH_INFO: `/v3/users/3bd5d96ebc5042748373951a9d4120e6` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:28.331658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd5d96ebc5042748373951a9d4120e6)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:28.333702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:28.382330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd5d96ebc5042748373951a9d4120e6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.394753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd5d96ebc5042748373951a9d4120e6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.395539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd5d96ebc5042748373951a9d4120e6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.404940 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd5d96ebc5042748373951a9d4120e6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.418040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-61c43813-0093-4f3a-9646-a845a4802f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd5d96ebc5042748373951a9d4120e6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.424442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1191/2382] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:22:28 2022] DELETE /identity/v3/users/3bd5d96ebc5042748373951a9d4120e6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:28.716226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:28.717700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:28.730941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cf5cd-2d4a-418d-aa68-eeba3ba85b28 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:28.790502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:28.791422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:28.791827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:28.792126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] PATH_INFO: `/v3/projects/8c05ccf515864b3395e7b2b367036796` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:28.793371 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c05ccf515864b3395e7b2b367036796)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:28.795343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:28.843735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c05ccf515864b3395e7b2b367036796', 'request_id': 'req-498d44f9-ee2f-452f-a396-d31b5ac05d93'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.850363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c05ccf515864b3395e7b2b367036796', 'request_id': 'req-498d44f9-ee2f-452f-a396-d31b5ac05d93'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.856258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c05ccf515864b3395e7b2b367036796', 'request_id': 'req-498d44f9-ee2f-452f-a396-d31b5ac05d93'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.871434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] The token cache is being invalidate because project 8c05ccf515864b3395e7b2b367036796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:22:28.871839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.872468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-498d44f9-ee2f-452f-a396-d31b5ac05d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:22:28.874063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1192/2383] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:22:28 2022] DELETE /identity/v3/projects/8c05ccf515864b3395e7b2b367036796 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:28.879199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ae8f-6fae-4bf3-b488-766ea3c63a54 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:28.879747 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ae8f-6fae-4bf3-b488-766ea3c63a54 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:28.879747 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ae8f-6fae-4bf3-b488-766ea3c63a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:28.902947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-3016ae8f-6fae-4bf3-b488-766ea3c63a54 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:22:28.904280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3016ae8f-6fae-4bf3-b488-766ea3c63a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:28.962834 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1192/2384] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:22:28.970860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d67b9-4ab0-472e-9e37-41cbda3c94a9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:28.971151 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d67b9-4ab0-472e-9e37-41cbda3c94a9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:28.971463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d67b9-4ab0-472e-9e37-41cbda3c94a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:28.990506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c24d67b9-4ab0-472e-9e37-41cbda3c94a9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:22:28.991842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c24d67b9-4ab0-472e-9e37-41cbda3c94a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:29.001121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1193/2385] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:22:29.005470 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016ae8f-6fae-4bf3-b488-766ea3c63a54 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.016061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016ae8f-6fae-4bf3-b488-766ea3c63a54 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.022344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81909442-85c9-4298-a01a-07a8b23a29cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.023004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81909442-85c9-4298-a01a-07a8b23a29cb admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.023279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81909442-85c9-4298-a01a-07a8b23a29cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.023569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81909442-85c9-4298-a01a-07a8b23a29cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.024301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81909442-85c9-4298-a01a-07a8b23a29cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:22:29.024636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81909442-85c9-4298-a01a-07a8b23a29cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.026256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81909442-85c9-4298-a01a-07a8b23a29cb admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.039160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1193/2386] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 06:22:29.043830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d67b9-4ab0-472e-9e37-41cbda3c94a9 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.053689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d67b9-4ab0-472e-9e37-41cbda3c94a9 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.060324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a23cc-af78-4dd8-9861-7f741d8f4c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.060982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a23cc-af78-4dd8-9861-7f741d8f4c26 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.061268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a23cc-af78-4dd8-9861-7f741d8f4c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.061538 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a23cc-af78-4dd8-9861-7f741d8f4c26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.062252 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a23cc-af78-4dd8-9861-7f741d8f4c26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.064299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a23cc-af78-4dd8-9861-7f741d8f4c26 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.068720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:29.095457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1194/2387] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:22:29 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:22:29.100022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81909442-85c9-4298-a01a-07a8b23a29cb admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.110343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81909442-85c9-4298-a01a-07a8b23a29cb admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.168590 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f8652-03ce-442b-b529-0cb5b9e98d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.169262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f8652-03ce-442b-b529-0cb5b9e98d71 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.169542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f8652-03ce-442b-b529-0cb5b9e98d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.169816 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f8652-03ce-442b-b529-0cb5b9e98d71 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.170517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f8652-03ce-442b-b529-0cb5b9e98d71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.172472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f8652-03ce-442b-b529-0cb5b9e98d71 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.180286 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:29.196079 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1194/2388] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:22:29 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:22:29.200069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a23cc-af78-4dd8-9861-7f741d8f4c26 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.210848 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a23cc-af78-4dd8-9861-7f741d8f4c26 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.216104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e4c59-3cb9-4bac-b304-da7a8d5b4de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.216775 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e4c59-3cb9-4bac-b304-da7a8d5b4de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.217061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e4c59-3cb9-4bac-b304-da7a8d5b4de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.217336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e4c59-3cb9-4bac-b304-da7a8d5b4de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.218070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e4c59-3cb9-4bac-b304-da7a8d5b4de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.219732 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e4c59-3cb9-4bac-b304-da7a8d5b4de1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.230599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1195/2389] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:22:29.235539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f8652-03ce-442b-b529-0cb5b9e98d71 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.250466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f8652-03ce-442b-b529-0cb5b9e98d71 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.256121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3a45c-e668-4a84-b91b-653c565705c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.256978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3a45c-e668-4a84-b91b-653c565705c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.257270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3a45c-e668-4a84-b91b-653c565705c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.257543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3a45c-e668-4a84-b91b-653c565705c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.258278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3a45c-e668-4a84-b91b-653c565705c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.259938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3a45c-e668-4a84-b91b-653c565705c1 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.271549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1195/2390] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:22:29.275777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e4c59-3cb9-4bac-b304-da7a8d5b4de1 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.286638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e4c59-3cb9-4bac-b304-da7a8d5b4de1 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.292624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25284111-a325-4a75-a5e2-82a4b467ded0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.293451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25284111-a325-4a75-a5e2-82a4b467ded0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.293739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25284111-a325-4a75-a5e2-82a4b467ded0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.294012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25284111-a325-4a75-a5e2-82a4b467ded0 admin admin] PATH_INFO: `/v3/projects/c7f87e20614b42f783cbde35dca7b4e3/users/c96b5936369b43f1859f4352fa4dd5cb/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.310208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25284111-a325-4a75-a5e2-82a4b467ded0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7f87e20614b42f783cbde35dca7b4e3, user_id=c96b5936369b43f1859f4352fa4dd5cb, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.312930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25284111-a325-4a75-a5e2-82a4b467ded0 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.332363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1196/2391] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:22:29 2022] PUT /identity/v3/projects/c7f87e20614b42f783cbde35dca7b4e3/users/c96b5936369b43f1859f4352fa4dd5cb/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:22:29.343911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdfd4d-a105-4164-bca5-0b1500e596a7 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.344277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdfd4d-a105-4164-bca5-0b1500e596a7 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.344644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdfd4d-a105-4164-bca5-0b1500e596a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.372868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-f9fdfd4d-a105-4164-bca5-0b1500e596a7 None None] MFA Rules not processed for user `c96b5936369b43f1859f4352fa4dd5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:22:29.374097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f9fdfd4d-a105-4164-bca5-0b1500e596a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:29.441454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1196/2392] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:22:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:22:29.446886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25284111-a325-4a75-a5e2-82a4b467ded0 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.521342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25284111-a325-4a75-a5e2-82a4b467ded0 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.526389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb710a1f-d5a7-4410-acf5-a48a07aefc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.527096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb710a1f-d5a7-4410-acf5-a48a07aefc7d admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.527410 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb710a1f-d5a7-4410-acf5-a48a07aefc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.527688 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb710a1f-d5a7-4410-acf5-a48a07aefc7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.528386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb710a1f-d5a7-4410-acf5-a48a07aefc7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.530304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb710a1f-d5a7-4410-acf5-a48a07aefc7d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.557397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1197/2393] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:22:29 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:22:29.561643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdfd4d-a105-4164-bca5-0b1500e596a7 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.573070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdfd4d-a105-4164-bca5-0b1500e596a7 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.629667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a2ee1-de4f-47aa-b3de-482aae6ed9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.630337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a2ee1-de4f-47aa-b3de-482aae6ed9bb admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.630621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a2ee1-de4f-47aa-b3de-482aae6ed9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.630895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a2ee1-de4f-47aa-b3de-482aae6ed9bb admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.631624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a2ee1-de4f-47aa-b3de-482aae6ed9bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.633554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a2ee1-de4f-47aa-b3de-482aae6ed9bb admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.655719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1197/2394] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:22:29 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:22:29.660937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb710a1f-d5a7-4410-acf5-a48a07aefc7d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.671831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb710a1f-d5a7-4410-acf5-a48a07aefc7d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.676668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd8baa-5bfb-4995-9eba-fdcdc813b512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.677329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8baa-5bfb-4995-9eba-fdcdc813b512 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.677630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8baa-5bfb-4995-9eba-fdcdc813b512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.677904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8baa-5bfb-4995-9eba-fdcdc813b512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.678636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd8baa-5bfb-4995-9eba-fdcdc813b512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.680285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd8baa-5bfb-4995-9eba-fdcdc813b512 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.691058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1198/2395] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:22:29.695636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a2ee1-de4f-47aa-b3de-482aae6ed9bb admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.706104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a2ee1-de4f-47aa-b3de-482aae6ed9bb admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.711480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c54c0e-74b2-4154-9673-d52af57f6445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.712453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c54c0e-74b2-4154-9673-d52af57f6445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.712804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c54c0e-74b2-4154-9673-d52af57f6445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.713159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c54c0e-74b2-4154-9673-d52af57f6445 admin admin] PATH_INFO: `/v3/projects/64989f684c3540e59c377f060755a5d1/users/e1223eb664e04ffb991b30650fa80f12/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.728320 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c54c0e-74b2-4154-9673-d52af57f6445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64989f684c3540e59c377f060755a5d1, user_id=e1223eb664e04ffb991b30650fa80f12, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.730990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c54c0e-74b2-4154-9673-d52af57f6445 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.753916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1198/2396] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:22:29 2022] PUT /identity/v3/projects/64989f684c3540e59c377f060755a5d1/users/e1223eb664e04ffb991b30650fa80f12/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:29.758097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd8baa-5bfb-4995-9eba-fdcdc813b512 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.770965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd8baa-5bfb-4995-9eba-fdcdc813b512 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.834326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2de16-d8ae-4913-a33c-f1e2b72793f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.836041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2de16-d8ae-4913-a33c-f1e2b72793f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.836325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2de16-d8ae-4913-a33c-f1e2b72793f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.836606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2de16-d8ae-4913-a33c-f1e2b72793f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.837344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2de16-d8ae-4913-a33c-f1e2b72793f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.838956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2de16-d8ae-4913-a33c-f1e2b72793f1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.852891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1199/2397] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:22:29.854392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c54c0e-74b2-4154-9673-d52af57f6445 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:22:29.865186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c54c0e-74b2-4154-9673-d52af57f6445 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:22:29.870252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc81724-1508-43fb-a4d0-3ca28740b3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:22:29.871149 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81724-1508-43fb-a4d0-3ca28740b3b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.871465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81724-1508-43fb-a4d0-3ca28740b3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.871734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81724-1508-43fb-a4d0-3ca28740b3b4 admin admin] PATH_INFO: `/v3/projects/64989f684c3540e59c377f060755a5d1/users/e1223eb664e04ffb991b30650fa80f12/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.873719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc81724-1508-43fb-a4d0-3ca28740b3b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64989f684c3540e59c377f060755a5d1, user_id=e1223eb664e04ffb991b30650fa80f12, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:22:29.876501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc81724-1508-43fb-a4d0-3ca28740b3b4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:22:29.887583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1199/2398] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:22:29 2022] PUT /identity/v3/projects/64989f684c3540e59c377f060755a5d1/users/e1223eb664e04ffb991b30650fa80f12/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:22:29.953318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a7355-b3c7-4777-90b4-6f45d71a1e7a None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:22:29.953614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a7355-b3c7-4777-90b4-6f45d71a1e7a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:22:29.953894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a7355-b3c7-4777-90b4-6f45d71a1e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:22:29.980744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-5a8a7355-b3c7-4777-90b4-6f45d71a1e7a None None] MFA Rules not processed for user `e1223eb664e04ffb991b30650fa80f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:22:29.981958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5a8a7355-b3c7-4777-90b4-6f45d71a1e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:22:29.993723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:30.055418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1200/2399] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:22:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:22:39.883929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:INSERT=7,keystone:SELECT=83 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:22:40.052108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=24 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:25:24.535130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc81724-1508-43fb-a4d0-3ca28740b3b4 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:25:24.537161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7cc81724-1508-43fb-a4d0-3ca28740b3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:25:24.608081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc81724-1508-43fb-a4d0-3ca28740b3b4 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:25:24.612166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99786416-3979-4655-b6cf-67a43bdceec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:25:24.612809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99786416-3979-4655-b6cf-67a43bdceec9 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:25:24.613084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99786416-3979-4655-b6cf-67a43bdceec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:25:24.613354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99786416-3979-4655-b6cf-67a43bdceec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:25:24.615367 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-99786416-3979-4655-b6cf-67a43bdceec9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:25:24.681163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99786416-3979-4655-b6cf-67a43bdceec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:25:24.682750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99786416-3979-4655-b6cf-67a43bdceec9 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:25:24.692033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1200/2400] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:25:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:25:34.687903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=53 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:25:34.786850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a7355-b3c7-4777-90b4-6f45d71a1e7a None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:25:34.788514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-5a8a7355-b3c7-4777-90b4-6f45d71a1e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:25:34.870568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a7355-b3c7-4777-90b4-6f45d71a1e7a None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:25:34.874487 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499137d-715e-47c6-88f0-2a2d057be2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:25:34.875152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499137d-715e-47c6-88f0-2a2d057be2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:25:34.875462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499137d-715e-47c6-88f0-2a2d057be2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:25:34.875738 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499137d-715e-47c6-88f0-2a2d057be2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:25:34.877540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2499137d-715e-47c6-88f0-2a2d057be2d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:25:34.888855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499137d-715e-47c6-88f0-2a2d057be2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:25:34.890363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499137d-715e-47c6-88f0-2a2d057be2d1 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:25:34.898426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1201/2401] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:25:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:25:44.895073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=26 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:26:22.346040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99786416-3979-4655-b6cf-67a43bdceec9 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:26:22.356162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99786416-3979-4655-b6cf-67a43bdceec9 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:26:22.361181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb4985-7f45-4c56-a33a-973dc2be5ca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:26:22.361819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb4985-7f45-4c56-a33a-973dc2be5ca0 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:26:22.362097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb4985-7f45-4c56-a33a-973dc2be5ca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:26:22.362397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb4985-7f45-4c56-a33a-973dc2be5ca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:26:22.364669 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5ccb4985-7f45-4c56-a33a-973dc2be5ca0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:26:22.437691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb4985-7f45-4c56-a33a-973dc2be5ca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:26:22.439147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb4985-7f45-4c56-a33a-973dc2be5ca0 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:26:22.459006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1201/2402] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:26:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:26:23.606641 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499137d-715e-47c6-88f0-2a2d057be2d1 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:26:23.618702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499137d-715e-47c6-88f0-2a2d057be2d1 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:26:23.623213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c7271-6a98-40bf-b64e-3e4f9b27dc8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:26:23.623893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c7271-6a98-40bf-b64e-3e4f9b27dc8a service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:26:23.624169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c7271-6a98-40bf-b64e-3e4f9b27dc8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:26:23.624437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c7271-6a98-40bf-b64e-3e4f9b27dc8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:26:23.626589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-871c7271-6a98-40bf-b64e-3e4f9b27dc8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:26:23.690155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c7271-6a98-40bf-b64e-3e4f9b27dc8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:26:23.691665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c7271-6a98-40bf-b64e-3e4f9b27dc8a service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:26:23.701298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1202/2403] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:26:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:26:32.455985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=36 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:26:33.697074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=28 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:27:23.891602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb4985-7f45-4c56-a33a-973dc2be5ca0 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:27:23.900542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb4985-7f45-4c56-a33a-973dc2be5ca0 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:27:23.905036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4f9ed-7e9b-4eb9-83b0-740bf740a521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:27:23.905679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4f9ed-7e9b-4eb9-83b0-740bf740a521 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:27:23.905925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4f9ed-7e9b-4eb9-83b0-740bf740a521 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:27:23.906252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4f9ed-7e9b-4eb9-83b0-740bf740a521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:27:23.908202 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ced4f9ed-7e9b-4eb9-83b0-740bf740a521 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:27:24.005700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4f9ed-7e9b-4eb9-83b0-740bf740a521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:27:24.007157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4f9ed-7e9b-4eb9-83b0-740bf740a521 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:27:24.026858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1202/2404] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:27:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:27:29.079966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c7271-6a98-40bf-b64e-3e4f9b27dc8a service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:27:29.081483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-871c7271-6a98-40bf-b64e-3e4f9b27dc8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:27:29.207385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c7271-6a98-40bf-b64e-3e4f9b27dc8a service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:27:29.212785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b4924-f4f4-48cb-9d39-aa9fb457aaac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:27:29.213442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b4924-f4f4-48cb-9d39-aa9fb457aaac service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:27:29.213725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b4924-f4f4-48cb-9d39-aa9fb457aaac service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:27:29.214000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b4924-f4f4-48cb-9d39-aa9fb457aaac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:27:29.215990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e87b4924-f4f4-48cb-9d39-aa9fb457aaac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:27:29.228019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b4924-f4f4-48cb-9d39-aa9fb457aaac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:27:29.229609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b4924-f4f4-48cb-9d39-aa9fb457aaac service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:27:29.239645 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1203/2405] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:27:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:27:34.024327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=42 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:27:39.235797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=49 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:20.755604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4f9ed-7e9b-4eb9-83b0-740bf740a521 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:20.775751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4f9ed-7e9b-4eb9-83b0-740bf740a521 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:20.779824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b7a34-ce94-4068-bdd6-35dced3a9a03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:20.780496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b7a34-ce94-4068-bdd6-35dced3a9a03 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:20.780782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b7a34-ce94-4068-bdd6-35dced3a9a03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:20.781047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b7a34-ce94-4068-bdd6-35dced3a9a03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:20.782862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cb6b7a34-ce94-4068-bdd6-35dced3a9a03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:20.880586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b7a34-ce94-4068-bdd6-35dced3a9a03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:20.882050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b7a34-ce94-4068-bdd6-35dced3a9a03 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:20.902721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1203/2406] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:21.077003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b4924-f4f4-48cb-9d39-aa9fb457aaac service placement] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:21.078524 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e87b4924-f4f4-48cb-9d39-aa9fb457aaac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:21.112024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b4924-f4f4-48cb-9d39-aa9fb457aaac service placement] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:21.116531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d065b28-8d01-4d70-8733-bda1ab9891f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:21.117207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d065b28-8d01-4d70-8733-bda1ab9891f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:21.117486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d065b28-8d01-4d70-8733-bda1ab9891f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:21.117754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d065b28-8d01-4d70-8733-bda1ab9891f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:21.119918 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6d065b28-8d01-4d70-8733-bda1ab9891f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:21.203962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d065b28-8d01-4d70-8733-bda1ab9891f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:21.205339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d065b28-8d01-4d70-8733-bda1ab9891f3 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:21.225873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1204/2407] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:21.455603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b7a34-ce94-4068-bdd6-35dced3a9a03 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:21.465838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b7a34-ce94-4068-bdd6-35dced3a9a03 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:21.470359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176d186-47fb-4b02-ab3c-e1c199aaeabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:21.471072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176d186-47fb-4b02-ab3c-e1c199aaeabb service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:21.471482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176d186-47fb-4b02-ab3c-e1c199aaeabb service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:21.471797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176d186-47fb-4b02-ab3c-e1c199aaeabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:21.473928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e176d186-47fb-4b02-ab3c-e1c199aaeabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:21.564847 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176d186-47fb-4b02-ab3c-e1c199aaeabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:21.566300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176d186-47fb-4b02-ab3c-e1c199aaeabb service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:21.577565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1204/2408] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:21.798661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d065b28-8d01-4d70-8733-bda1ab9891f3 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:21.808424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d065b28-8d01-4d70-8733-bda1ab9891f3 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:21.812466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cecef-c30c-4b7d-9c80-00edd591af4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:21.813235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cecef-c30c-4b7d-9c80-00edd591af4b service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:21.813543 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cecef-c30c-4b7d-9c80-00edd591af4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:21.813823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cecef-c30c-4b7d-9c80-00edd591af4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:21.815667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-754cecef-c30c-4b7d-9c80-00edd591af4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:21.881699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754cecef-c30c-4b7d-9c80-00edd591af4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:21.883176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754cecef-c30c-4b7d-9c80-00edd591af4b service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:21.893277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1205/2409] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:22.763662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176d186-47fb-4b02-ab3c-e1c199aaeabb service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:22.765140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e176d186-47fb-4b02-ab3c-e1c199aaeabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:22.777695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176d186-47fb-4b02-ab3c-e1c199aaeabb service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:22.782373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adea177-02e6-4ad3-9fa6-8db9c304473b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:22.783037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adea177-02e6-4ad3-9fa6-8db9c304473b service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:22.783562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adea177-02e6-4ad3-9fa6-8db9c304473b service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:22.783881 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adea177-02e6-4ad3-9fa6-8db9c304473b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:22.785879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2adea177-02e6-4ad3-9fa6-8db9c304473b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:22.804213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adea177-02e6-4ad3-9fa6-8db9c304473b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:22.805710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adea177-02e6-4ad3-9fa6-8db9c304473b service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:22.820340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1205/2410] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:25.105024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cecef-c30c-4b7d-9c80-00edd591af4b service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:25.106676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-754cecef-c30c-4b7d-9c80-00edd591af4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:25.192547 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cecef-c30c-4b7d-9c80-00edd591af4b service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:25.223492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2063c53-4594-4fd4-83f9-077d9e772fa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:25.225070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2063c53-4594-4fd4-83f9-077d9e772fa6 service nova] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:25.225358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2063c53-4594-4fd4-83f9-077d9e772fa6 service nova] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:25.225636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2063c53-4594-4fd4-83f9-077d9e772fa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:25.231886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a2063c53-4594-4fd4-83f9-077d9e772fa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:25.281589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2063c53-4594-4fd4-83f9-077d9e772fa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:25.287010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2063c53-4594-4fd4-83f9-077d9e772fa6 service nova] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:25.316978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1206/2411] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:25.400848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adea177-02e6-4ad3-9fa6-8db9c304473b service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:25.402336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2adea177-02e6-4ad3-9fa6-8db9c304473b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:25.423136 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adea177-02e6-4ad3-9fa6-8db9c304473b service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:25.439663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeb396-0624-4127-b308-63d276d3d5b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:25.440541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb396-0624-4127-b308-63d276d3d5b1 service placement] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:25.442160 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb396-0624-4127-b308-63d276d3d5b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:25.444172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb396-0624-4127-b308-63d276d3d5b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:25.449344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-abbeb396-0624-4127-b308-63d276d3d5b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:25.467205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeb396-0624-4127-b308-63d276d3d5b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:25.469290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeb396-0624-4127-b308-63d276d3d5b1 service placement] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:25.483787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1206/2412] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:25.534376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2063c53-4594-4fd4-83f9-077d9e772fa6 service nova] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:25.536376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a2063c53-4594-4fd4-83f9-077d9e772fa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:25.567534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2063c53-4594-4fd4-83f9-077d9e772fa6 service nova] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:25.573391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:25.574480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:25.574770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:25.575047 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] PATH_INFO: `/v3/users/e367c76dfa7f4d2791e60ce148f352c3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:25.576497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e367c76dfa7f4d2791e60ce148f352c3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:25.579676 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:25.618769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:25.654435 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e367c76dfa7f4d2791e60ce148f352c3'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:25.662431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e367c76dfa7f4d2791e60ce148f352c3'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:25.675559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e367c76dfa7f4d2791e60ce148f352c3'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:25.676277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e367c76dfa7f4d2791e60ce148f352c3'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:25.677308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e367c76dfa7f4d2791e60ce148f352c3'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:25.704132 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1207/2413] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:25 2022] DELETE /identity/v3/users/e367c76dfa7f4d2791e60ce148f352c3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:25.882755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeb396-0624-4127-b308-63d276d3d5b1 service placement] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:25.889392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:25.900207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:25.902599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:25.921086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeb396-0624-4127-b308-63d276d3d5b1 service placement] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:25.996686 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03855d5d-d196-47f0-bc96-7f1264d8f566 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.000339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4ab47-cec3-4d9c-a168-66e5cfdcf9f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.001357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4ab47-cec3-4d9c-a168-66e5cfdcf9f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.001864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4ab47-cec3-4d9c-a168-66e5cfdcf9f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.002215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.002615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4ab47-cec3-4d9c-a168-66e5cfdcf9f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.003465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.004191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.004675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] PATH_INFO: `/v3/projects/4e16375231df4b768b8355e320e91731` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.006154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e16375231df4b768b8355e320e91731)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.006390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-58b4ab47-cec3-4d9c-a168-66e5cfdcf9f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:26.008477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.077782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e16375231df4b768b8355e320e91731', 'request_id': 'req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:26.084595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e16375231df4b768b8355e320e91731', 'request_id': 'req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:26.090896 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e16375231df4b768b8355e320e91731', 'request_id': 'req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:26.104653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4ab47-cec3-4d9c-a168-66e5cfdcf9f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.106769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4ab47-cec3-4d9c-a168-66e5cfdcf9f1 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.107018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] The token cache is being invalidate because project 4e16375231df4b768b8355e320e91731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:26.107504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:26.108058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b26d8e8-c6db-4ff5-a997-f9f7d4ec00e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:26.110322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1208/2414] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:25 2022] DELETE /identity/v3/projects/4e16375231df4b768b8355e320e91731 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:26.117605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8e7a5-baa1-44ac-be77-ca62badbeb4a None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.118069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8e7a5-baa1-44ac-be77-ca62badbeb4a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.118494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8e7a5-baa1-44ac-be77-ca62badbeb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.123043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1207/2415] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:26.173443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-0fb8e7a5-baa1-44ac-be77-ca62badbeb4a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:26.174838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0fb8e7a5-baa1-44ac-be77-ca62badbeb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:26.181152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:26.240322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1209/2416] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:26.248131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa265e-0aea-4f84-8167-8bcaddac6195 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.248421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa265e-0aea-4f84-8167-8bcaddac6195 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.248698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa265e-0aea-4f84-8167-8bcaddac6195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.267378 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-f1aa265e-0aea-4f84-8167-8bcaddac6195 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:26.268587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f1aa265e-0aea-4f84-8167-8bcaddac6195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:26.276742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1208/2417] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:26.281039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8e7a5-baa1-44ac-be77-ca62badbeb4a None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.291780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8e7a5-baa1-44ac-be77-ca62badbeb4a None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.302586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cd1e1-7a99-45f9-af56-4df17a97f711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.303436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cd1e1-7a99-45f9-af56-4df17a97f711 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.303734 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cd1e1-7a99-45f9-af56-4df17a97f711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.303992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cd1e1-7a99-45f9-af56-4df17a97f711 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.304736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cd1e1-7a99-45f9-af56-4df17a97f711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:26.305059 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cd1e1-7a99-45f9-af56-4df17a97f711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.306655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cd1e1-7a99-45f9-af56-4df17a97f711 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.318852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1210/2418] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:29:26.323528 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa265e-0aea-4f84-8167-8bcaddac6195 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.333205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa265e-0aea-4f84-8167-8bcaddac6195 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.338816 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f359c0-9dc6-4acc-8a28-c8064de21c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.339505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f359c0-9dc6-4acc-8a28-c8064de21c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.339782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f359c0-9dc6-4acc-8a28-c8064de21c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.340061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f359c0-9dc6-4acc-8a28-c8064de21c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.340743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f359c0-9dc6-4acc-8a28-c8064de21c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.342654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f359c0-9dc6-4acc-8a28-c8064de21c68 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.383358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1209/2419] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:26 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:26.387238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cd1e1-7a99-45f9-af56-4df17a97f711 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.397368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cd1e1-7a99-45f9-af56-4df17a97f711 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.452860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f359c0-9dc6-4acc-8a28-c8064de21c68 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.454647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a5f359c0-9dc6-4acc-8a28-c8064de21c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:26.457911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6ec71-dc61-4a25-997f-1e8456a99eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.458596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6ec71-dc61-4a25-997f-1e8456a99eee admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.458881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6ec71-dc61-4a25-997f-1e8456a99eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.459146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6ec71-dc61-4a25-997f-1e8456a99eee admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.459872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6ec71-dc61-4a25-997f-1e8456a99eee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.461787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6ec71-dc61-4a25-997f-1e8456a99eee admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.487153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1211/2420] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:26 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:26.491445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6ec71-dc61-4a25-997f-1e8456a99eee admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.501443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6ec71-dc61-4a25-997f-1e8456a99eee admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.506716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756c95d-b2b6-41e3-b5d0-3dbd5c5f38cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.507439 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756c95d-b2b6-41e3-b5d0-3dbd5c5f38cd admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.507722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756c95d-b2b6-41e3-b5d0-3dbd5c5f38cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.507993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756c95d-b2b6-41e3-b5d0-3dbd5c5f38cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.508757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756c95d-b2b6-41e3-b5d0-3dbd5c5f38cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.510366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756c95d-b2b6-41e3-b5d0-3dbd5c5f38cd admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.525292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1212/2421] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:26.528482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f359c0-9dc6-4acc-8a28-c8064de21c68 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.529511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756c95d-b2b6-41e3-b5d0-3dbd5c5f38cd admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.533037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048be0e-4ef0-41cb-bea6-a864c97cd2a2 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1223eb664e04ffb991b30650fa80f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64989f684c3540e59c377f060755a5d1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.533706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048be0e-4ef0-41cb-bea6-a864c97cd2a2 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.534022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048be0e-4ef0-41cb-bea6-a864c97cd2a2 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.534299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048be0e-4ef0-41cb-bea6-a864c97cd2a2 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.535043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048be0e-4ef0-41cb-bea6-a864c97cd2a2 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:26.535388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048be0e-4ef0-41cb-bea6-a864c97cd2a2 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.537017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048be0e-4ef0-41cb-bea6-a864c97cd2a2 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.539179 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756c95d-b2b6-41e3-b5d0-3dbd5c5f38cd admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.543810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e78918-90e8-433a-a741-a49f652e4199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.544463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e78918-90e8-433a-a741-a49f652e4199 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.544751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e78918-90e8-433a-a741-a49f652e4199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.545028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e78918-90e8-433a-a741-a49f652e4199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.545753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e78918-90e8-433a-a741-a49f652e4199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.547502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e78918-90e8-433a-a741-a49f652e4199 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.549242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1210/2422] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 06:29:26.553401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048be0e-4ef0-41cb-bea6-a864c97cd2a2 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.562966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1213/2423] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:26.563673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:26.564249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e78918-90e8-433a-a741-a49f652e4199 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.564483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048be0e-4ef0-41cb-bea6-a864c97cd2a2 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.570583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3377d-d892-4539-b8a0-6b9a398ae89d tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1223eb664e04ffb991b30650fa80f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64989f684c3540e59c377f060755a5d1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.571290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3377d-d892-4539-b8a0-6b9a398ae89d tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.571585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3377d-d892-4539-b8a0-6b9a398ae89d tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.572407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3377d-d892-4539-b8a0-6b9a398ae89d tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.573033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3377d-d892-4539-b8a0-6b9a398ae89d tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.574827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e78918-90e8-433a-a741-a49f652e4199 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.574997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3377d-d892-4539-b8a0-6b9a398ae89d tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.580333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf29286-f753-4768-851e-8531af8474bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.581252 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf29286-f753-4768-851e-8531af8474bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.581541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf29286-f753-4768-851e-8531af8474bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.581819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf29286-f753-4768-851e-8531af8474bf admin admin] PATH_INFO: `/v3/projects/26ae257e47dc48b69a5f81d22e9733e0/users/22eeb910338e4e5f8f761a1ff2557c5a/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.598824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf29286-f753-4768-851e-8531af8474bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ae257e47dc48b69a5f81d22e9733e0, user_id=22eeb910338e4e5f8f761a1ff2557c5a, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.601716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf29286-f753-4768-851e-8531af8474bf admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.611276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1211/2424] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:26 2022] POST /identity/v3/projects => generated 354 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:26.615143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3377d-d892-4539-b8a0-6b9a398ae89d tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.620241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1214/2425] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:26 2022] PUT /identity/v3/projects/26ae257e47dc48b69a5f81d22e9733e0/users/22eeb910338e4e5f8f761a1ff2557c5a/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:26.630985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02a4c4-1a30-4916-a2e3-602ac7eb8fbf None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.631499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02a4c4-1a30-4916-a2e3-602ac7eb8fbf None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.631946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02a4c4-1a30-4916-a2e3-602ac7eb8fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.662077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4b02a4c4-1a30-4916-a2e3-602ac7eb8fbf None None] MFA Rules not processed for user `22eeb910338e4e5f8f761a1ff2557c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:26.663394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4b02a4c4-1a30-4916-a2e3-602ac7eb8fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:26.696038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3377d-d892-4539-b8a0-6b9a398ae89d tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.701076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:26.749825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1215/2426] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:26.754619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02a4c4-1a30-4916-a2e3-602ac7eb8fbf None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.773213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04cdd9-b532-4e2a-8d06-e958a5c40eba tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1223eb664e04ffb991b30650fa80f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64989f684c3540e59c377f060755a5d1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.773989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04cdd9-b532-4e2a-8d06-e958a5c40eba tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.774695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04cdd9-b532-4e2a-8d06-e958a5c40eba tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.774983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04cdd9-b532-4e2a-8d06-e958a5c40eba tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.775750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04cdd9-b532-4e2a-8d06-e958a5c40eba tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:26.776359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04cdd9-b532-4e2a-8d06-e958a5c40eba tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.778050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04cdd9-b532-4e2a-8d06-e958a5c40eba tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.791165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1212/2427] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 06:29:26.827978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02a4c4-1a30-4916-a2e3-602ac7eb8fbf None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.835114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04cdd9-b532-4e2a-8d06-e958a5c40eba tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.837031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a455bb-035e-42e0-b9d3-059fd8d5d1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.837748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a455bb-035e-42e0-b9d3-059fd8d5d1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.838023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a455bb-035e-42e0-b9d3-059fd8d5d1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.838264 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a455bb-035e-42e0-b9d3-059fd8d5d1f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.839340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a455bb-035e-42e0-b9d3-059fd8d5d1f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.841497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a455bb-035e-42e0-b9d3-059fd8d5d1f2 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.848267 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04cdd9-b532-4e2a-8d06-e958a5c40eba tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.853360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1223eb664e04ffb991b30650fa80f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64989f684c3540e59c377f060755a5d1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:26.854380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:26.854792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:26.855141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] PATH_INFO: `/v3/projects/ab09a1c4d2644a28a97de567718be982` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:26.856430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ab09a1c4d2644a28a97de567718be982)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:26.858563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:26.872979 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1216/2428] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:26 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:26.876662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a455bb-035e-42e0-b9d3-059fd8d5d1f2 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:26.921162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab09a1c4d2644a28a97de567718be982', 'request_id': 'req-f61c772f-d0f1-4c33-87c6-90666f21dde4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:26.922447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab09a1c4d2644a28a97de567718be982', 'request_id': 'req-f61c772f-d0f1-4c33-87c6-90666f21dde4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:26.928326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab09a1c4d2644a28a97de567718be982', 'request_id': 'req-f61c772f-d0f1-4c33-87c6-90666f21dde4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:26.942530 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] The token cache is being invalidate because project ab09a1c4d2644a28a97de567718be982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:26.942956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:26.943674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:26.943777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a455bb-035e-42e0-b9d3-059fd8d5d1f2 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:26.945403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1213/2429] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:26 2022] DELETE /identity/v3/projects/ab09a1c4d2644a28a97de567718be982 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:27.001354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762a7a2-fcaa-4661-a721-b7827d26fda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:27.002165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762a7a2-fcaa-4661-a721-b7827d26fda4 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.002568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762a7a2-fcaa-4661-a721-b7827d26fda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.002919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762a7a2-fcaa-4661-a721-b7827d26fda4 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.004064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762a7a2-fcaa-4661-a721-b7827d26fda4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:27.006222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762a7a2-fcaa-4661-a721-b7827d26fda4 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:27.015668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:27.029851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1217/2430] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:26 2022] POST /identity/v3/users => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:27.033759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:27.035371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.045514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:27.051347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c772f-d0f1-4c33-87c6-90666f21dde4 tempest-VolumeQuotaClassesTest-724495793 tempest-VolumeQuotaClassesTest-724495793-project-admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:27.055442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762a7a2-fcaa-4661-a721-b7827d26fda4 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:27.056927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3137b-7d25-4af8-aa4a-db96e687fd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:27.057473 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3137b-7d25-4af8-aa4a-db96e687fd39 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.057593 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c762a7a2-fcaa-4661-a721-b7827d26fda4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.057760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3137b-7d25-4af8-aa4a-db96e687fd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.058360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3137b-7d25-4af8-aa4a-db96e687fd39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.059684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3137b-7d25-4af8-aa4a-db96e687fd39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:27.061639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3137b-7d25-4af8-aa4a-db96e687fd39 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:27.070278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762a7a2-fcaa-4661-a721-b7827d26fda4 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:27.073141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1214/2431] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:27.076928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:27.077234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3137b-7d25-4af8-aa4a-db96e687fd39 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:27.077481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.077813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.078041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] PATH_INFO: `/v3/users/9a963453dec24163ada4a51322997303` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.096231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3137b-7d25-4af8-aa4a-db96e687fd39 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:27.105017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a963453dec24163ada4a51322997303)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:27.106950 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ceca0-f836-4e99-8905-ee4af5f7bae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:27.107885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ceca0-f836-4e99-8905-ee4af5f7bae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.108321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ceca0-f836-4e99-8905-ee4af5f7bae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.108430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ceca0-f836-4e99-8905-ee4af5f7bae4 admin admin] PATH_INFO: `/v3/projects/9eb96babd12848e8bd45ee7c232dc9cc/users/f1709f21864e47dc8c3a2367f1789299/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.124392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:27.125048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ceca0-f836-4e99-8905-ee4af5f7bae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb96babd12848e8bd45ee7c232dc9cc, user_id=f1709f21864e47dc8c3a2367f1789299, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:27.130888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ceca0-f836-4e99-8905-ee4af5f7bae4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:27.153223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1215/2432] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:27 2022] PUT /identity/v3/projects/9eb96babd12848e8bd45ee7c232dc9cc/users/f1709f21864e47dc8c3a2367f1789299/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:27.156021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ceca0-f836-4e99-8905-ee4af5f7bae4 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:27.214075 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a963453dec24163ada4a51322997303'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:27.225117 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a963453dec24163ada4a51322997303'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:27.236185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a963453dec24163ada4a51322997303'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:27.237443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a963453dec24163ada4a51322997303'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:27.238404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a963453dec24163ada4a51322997303'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:27.257106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ceca0-f836-4e99-8905-ee4af5f7bae4 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:27.262634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076ffe8-1175-4e64-8b46-273218152d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:27.263498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076ffe8-1175-4e64-8b46-273218152d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.263799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076ffe8-1175-4e64-8b46-273218152d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.264074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076ffe8-1175-4e64-8b46-273218152d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.264981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076ffe8-1175-4e64-8b46-273218152d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:27.266673 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076ffe8-1175-4e64-8b46-273218152d03 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:27.268976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1218/2433] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:27 2022] DELETE /identity/v3/users/9a963453dec24163ada4a51322997303 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:27.272224 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:27.277090 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.282413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1216/2434] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:27.287298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3096-c7a6-4f1e-a388-fc294eb369fb None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.287548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3096-c7a6-4f1e-a388-fc294eb369fb None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.287793 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3096-c7a6-4f1e-a388-fc294eb369fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.309487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-756d3096-c7a6-4f1e-a388-fc294eb369fb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:27.310957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-756d3096-c7a6-4f1e-a388-fc294eb369fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.366598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac4678-6285-4fab-8e49-aaf42d3bc53a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:27.372446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:27.373323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.373599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.373873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] PATH_INFO: `/v3/users/c96b5936369b43f1859f4352fa4dd5cb` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.375763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96b5936369b43f1859f4352fa4dd5cb)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:27.376765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1217/2435] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:27.377736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:27.377947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d3096-c7a6-4f1e-a388-fc294eb369fb None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:27.380042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-756d3096-c7a6-4f1e-a388-fc294eb369fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.394114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d3096-c7a6-4f1e-a388-fc294eb369fb None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:27.401011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dd8e3-8d40-419f-846b-6b56aab27340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:27.404121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dd8e3-8d40-419f-846b-6b56aab27340 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.404121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dd8e3-8d40-419f-846b-6b56aab27340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.404121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dd8e3-8d40-419f-846b-6b56aab27340 admin admin] PATH_INFO: `/v3/projects/9eb96babd12848e8bd45ee7c232dc9cc/users/f1709f21864e47dc8c3a2367f1789299/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.406246 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32dd8e3-8d40-419f-846b-6b56aab27340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb96babd12848e8bd45ee7c232dc9cc, user_id=f1709f21864e47dc8c3a2367f1789299, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:27.409220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32dd8e3-8d40-419f-846b-6b56aab27340 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:27.422286 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1218/2436] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:27 2022] PUT /identity/v3/projects/9eb96babd12848e8bd45ee7c232dc9cc/users/f1709f21864e47dc8c3a2367f1789299/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:27.423527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dd8e3-8d40-419f-846b-6b56aab27340 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:27.424851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e32dd8e3-8d40-419f-846b-6b56aab27340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.464382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96b5936369b43f1859f4352fa4dd5cb'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:27.476008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96b5936369b43f1859f4352fa4dd5cb'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:27.487559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:INSERT=2,keystone:SELECT=81,keystone:DELETE=17 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:27.492403 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96b5936369b43f1859f4352fa4dd5cb'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:27.493134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96b5936369b43f1859f4352fa4dd5cb'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:27.494215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-796c2d70-c643-406c-8b6c-fa89bd379f1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96b5936369b43f1859f4352fa4dd5cb'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:27.495739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dd8e3-8d40-419f-846b-6b56aab27340 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:27.501340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7904b9-8df5-4de3-bdb6-6669eeab9cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:27.502038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7904b9-8df5-4de3-bdb6-6669eeab9cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.503477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7904b9-8df5-4de3-bdb6-6669eeab9cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.504509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7904b9-8df5-4de3-bdb6-6669eeab9cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.506996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9f7904b9-8df5-4de3-bdb6-6669eeab9cd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.516353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:27.521222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1219/2437] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:27 2022] DELETE /identity/v3/users/c96b5936369b43f1859f4352fa4dd5cb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:27.525315 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896cda0-44cc-4f22-8327-3af2e1a41d7b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.526050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896cda0-44cc-4f22-8327-3af2e1a41d7b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.526050 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896cda0-44cc-4f22-8327-3af2e1a41d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.564997 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a896cda0-44cc-4f22-8327-3af2e1a41d7b None None] MFA Rules not processed for user `f1709f21864e47dc8c3a2367f1789299`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:27.566268 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a896cda0-44cc-4f22-8327-3af2e1a41d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.591772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7904b9-8df5-4de3-bdb6-6669eeab9cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:27.593660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7904b9-8df5-4de3-bdb6-6669eeab9cd6 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:27.605166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1219/2438] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:27.606762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4d085-098f-4ec6-8dd2-156b84e01912 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.607099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4d085-098f-4ec6-8dd2-156b84e01912 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.607541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4d085-098f-4ec6-8dd2-156b84e01912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.633088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6af4d085-098f-4ec6-8dd2-156b84e01912 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:27.635183 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6af4d085-098f-4ec6-8dd2-156b84e01912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.660284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1220/2439] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:27.672239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896cda0-44cc-4f22-8327-3af2e1a41d7b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:27.674018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a896cda0-44cc-4f22-8327-3af2e1a41d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.721533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1220/2440] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:27.749058 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4d085-098f-4ec6-8dd2-156b84e01912 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:27.750446 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6af4d085-098f-4ec6-8dd2-156b84e01912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:27.776056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896cda0-44cc-4f22-8327-3af2e1a41d7b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:27.797224 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01053db5-69cc-46a4-8505-80eac1fc9652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:27.799770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01053db5-69cc-46a4-8505-80eac1fc9652 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.800275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01053db5-69cc-46a4-8505-80eac1fc9652 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.801195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01053db5-69cc-46a4-8505-80eac1fc9652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.818086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4d085-098f-4ec6-8dd2-156b84e01912 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:27.821408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01053db5-69cc-46a4-8505-80eac1fc9652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:27.822120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d89721-0009-474e-aefe-2a944137b198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:27.822869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d89721-0009-474e-aefe-2a944137b198 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:27.823220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d89721-0009-474e-aefe-2a944137b198 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:27.823430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01053db5-69cc-46a4-8505-80eac1fc9652 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:27.823726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d89721-0009-474e-aefe-2a944137b198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:27.834131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d89721-0009-474e-aefe-2a944137b198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:27.835790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d89721-0009-474e-aefe-2a944137b198 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:27.847426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1221/2441] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 06:29:27.861935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1221/2442] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:27.996070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d89721-0009-474e-aefe-2a944137b198 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:28.006413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d89721-0009-474e-aefe-2a944137b198 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:28.011875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a8bb3-9d09-46aa-ad96-8413dd64d0b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:28.012778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a8bb3-9d09-46aa-ad96-8413dd64d0b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:28.013122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a8bb3-9d09-46aa-ad96-8413dd64d0b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:28.013521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a8bb3-9d09-46aa-ad96-8413dd64d0b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:28.015778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-426a8bb3-9d09-46aa-ad96-8413dd64d0b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:28.077004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:28.097169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a8bb3-9d09-46aa-ad96-8413dd64d0b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:28.099201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a8bb3-9d09-46aa-ad96-8413dd64d0b8 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:28.114026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1222/2443] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:28.370307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01053db5-69cc-46a4-8505-80eac1fc9652 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:28.372508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-01053db5-69cc-46a4-8505-80eac1fc9652 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:28.388972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01053db5-69cc-46a4-8505-80eac1fc9652 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:28.390174 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a8bb3-9d09-46aa-ad96-8413dd64d0b8 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:28.394716 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:28.395148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-426a8bb3-9d09-46aa-ad96-8413dd64d0b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:28.396003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:28.396421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:28.396863 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] PATH_INFO: `/v3/projects/c7f87e20614b42f783cbde35dca7b4e3` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:28.398341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7f87e20614b42f783cbde35dca7b4e3)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:28.401917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:28.436515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a8bb3-9d09-46aa-ad96-8413dd64d0b8 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:28.443010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:28.444261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:28.444706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:28.444983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] PATH_INFO: `/v3/projects/aac7de21b69441899866f7685df2e5c2` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:28.458380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac7de21b69441899866f7685df2e5c2)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:28.464970 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:28.477800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7f87e20614b42f783cbde35dca7b4e3', 'request_id': 'req-ba480027-4933-427c-bf92-02a52f513fd5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.498219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7f87e20614b42f783cbde35dca7b4e3', 'request_id': 'req-ba480027-4933-427c-bf92-02a52f513fd5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.513558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:INSERT=1,keystone:DELETE=8,keystone:SELECT=91 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:28.531878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac7de21b69441899866f7685df2e5c2', 'request_id': 'req-cc8b1639-a8c3-4942-9a5f-03fadb30f568'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.532386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7f87e20614b42f783cbde35dca7b4e3', 'request_id': 'req-ba480027-4933-427c-bf92-02a52f513fd5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.545079 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac7de21b69441899866f7685df2e5c2', 'request_id': 'req-cc8b1639-a8c3-4942-9a5f-03fadb30f568'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.552945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac7de21b69441899866f7685df2e5c2', 'request_id': 'req-cc8b1639-a8c3-4942-9a5f-03fadb30f568'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.562444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] The token cache is being invalidate because project c7f87e20614b42f783cbde35dca7b4e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:28.562745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.564205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.568829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1222/2444] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:28 2022] DELETE /identity/v3/projects/c7f87e20614b42f783cbde35dca7b4e3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:28.570202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:28.573655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:28.582929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] The token cache is being invalidate because project aac7de21b69441899866f7685df2e5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:28.583221 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.587357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.589835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1223/2445] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:28 2022] DELETE /identity/v3/projects/aac7de21b69441899866f7685df2e5c2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:28.593955 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:28.595429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:28.662468 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba480027-4933-427c-bf92-02a52f513fd5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:28.667915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:28.669369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:28.669611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:28.669858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] PATH_INFO: `/v3/users/e1223eb664e04ffb991b30650fa80f12` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:28.671285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1223eb664e04ffb991b30650fa80f12)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:28.673461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:28.675604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b1639-a8c3-4942-9a5f-03fadb30f568 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:28.680910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:28.682056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:28.682411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:28.682740 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] PATH_INFO: `/v3/users/960bff9db72145bfb3908e033f5b100f` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:28.684089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=960bff9db72145bfb3908e033f5b100f)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:28.686226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:28.726287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1223eb664e04ffb991b30650fa80f12'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.734563 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1223eb664e04ffb991b30650fa80f12'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.737665 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '960bff9db72145bfb3908e033f5b100f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.742915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1223eb664e04ffb991b30650fa80f12'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.743599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1223eb664e04ffb991b30650fa80f12'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.744261 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1223eb664e04ffb991b30650fa80f12'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.745073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '960bff9db72145bfb3908e033f5b100f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.745770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '960bff9db72145bfb3908e033f5b100f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.749377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:28.755564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '960bff9db72145bfb3908e033f5b100f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.764695 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1223/2446] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:28 2022] DELETE /identity/v3/users/e1223eb664e04ffb991b30650fa80f12 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:28.769855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '960bff9db72145bfb3908e033f5b100f'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:28.776269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1224/2447] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:28 2022] DELETE /identity/v3/users/960bff9db72145bfb3908e033f5b100f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:29.043500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.045107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:29.052362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.055467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:29.134562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2e5c8-8620-40de-9e08-a1b448727d7b admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.139005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:29.139921 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.140176 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.140444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] PATH_INFO: `/v3/projects/f7c483de7eb94f0daf588dd58b6b89d7` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.141629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c483de7eb94f0daf588dd58b6b89d7)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:29.143732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:29.149801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef26e9-f171-4639-8b87-64f9a21029bb admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.154885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:29.155835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.156148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.156422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] PATH_INFO: `/v3/projects/64989f684c3540e59c377f060755a5d1` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.157600 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64989f684c3540e59c377f060755a5d1)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:29.159556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:29.166204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:29.193191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c483de7eb94f0daf588dd58b6b89d7', 'request_id': 'req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.199748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c483de7eb94f0daf588dd58b6b89d7', 'request_id': 'req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.206290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c483de7eb94f0daf588dd58b6b89d7', 'request_id': 'req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.211800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64989f684c3540e59c377f060755a5d1', 'request_id': 'req-ff883d85-a753-4042-a79b-fc199732d966'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.218612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64989f684c3540e59c377f060755a5d1', 'request_id': 'req-ff883d85-a753-4042-a79b-fc199732d966'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.219502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] The token cache is being invalidate because project f7c483de7eb94f0daf588dd58b6b89d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:29.219790 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.220423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.222250 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1225/2448] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:29 2022] DELETE /identity/v3/projects/f7c483de7eb94f0daf588dd58b6b89d7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:29.223089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.224427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:29.227324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64989f684c3540e59c377f060755a5d1', 'request_id': 'req-ff883d85-a753-4042-a79b-fc199732d966'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.240163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] The token cache is being invalidate because project 64989f684c3540e59c377f060755a5d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:29.240455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.241079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.245987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1224/2449] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:29 2022] DELETE /identity/v3/projects/64989f684c3540e59c377f060755a5d1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:29.245987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.245987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:29.284807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b93cc-ce4c-42d7-8ea2-e82bdbb30ea9 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.316100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:29.316525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff883d85-a753-4042-a79b-fc199732d966 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.322046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:29.322980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.323342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.323628 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] PATH_INFO: `/v3/users/2c08d31c15914b0eb4d034dcc6d6f483` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.324970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c08d31c15914b0eb4d034dcc6d6f483)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:29.326995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:29.349030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764a24a-b069-4cf0-9cd8-e76b357610ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:29.349801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764a24a-b069-4cf0-9cd8-e76b357610ab service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.350152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764a24a-b069-4cf0-9cd8-e76b357610ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.350428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764a24a-b069-4cf0-9cd8-e76b357610ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.352681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2764a24a-b069-4cf0-9cd8-e76b357610ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:29.366038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764a24a-b069-4cf0-9cd8-e76b357610ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:29.367537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764a24a-b069-4cf0-9cd8-e76b357610ab service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:29.375349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c08d31c15914b0eb4d034dcc6d6f483'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.377490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1226/2450] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:29.379071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31555b7e-fb62-4028-a544-0c8bcde600a5 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.379335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31555b7e-fb62-4028-a544-0c8bcde600a5 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.379613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31555b7e-fb62-4028-a544-0c8bcde600a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.383156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c08d31c15914b0eb4d034dcc6d6f483'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.395014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c08d31c15914b0eb4d034dcc6d6f483'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.395757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c08d31c15914b0eb4d034dcc6d6f483'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.396374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c08d31c15914b0eb4d034dcc6d6f483'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.398296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-31555b7e-fb62-4028-a544-0c8bcde600a5 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:29.399560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-31555b7e-fb62-4028-a544-0c8bcde600a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:29.408041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1227/2451] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:29.415333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f717765-e504-4b14-8c11-16da459d85a3 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.415624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f717765-e504-4b14-8c11-16da459d85a3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.415719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1225/2452] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:29 2022] DELETE /identity/v3/users/2c08d31c15914b0eb4d034dcc6d6f483 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:29.415953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f717765-e504-4b14-8c11-16da459d85a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.435151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6f717765-e504-4b14-8c11-16da459d85a3 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:29.436384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6f717765-e504-4b14-8c11-16da459d85a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:29.510712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1228/2453] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:29.515375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.532208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b09f65-8800-4787-a083-25d43e87f980 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.544790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6897bba-5374-474a-ace2-3e7edbae34ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:29.545498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6897bba-5374-474a-ace2-3e7edbae34ed admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.545781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6897bba-5374-474a-ace2-3e7edbae34ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.546057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6897bba-5374-474a-ace2-3e7edbae34ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.547818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6897bba-5374-474a-ace2-3e7edbae34ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:29.547818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6897bba-5374-474a-ace2-3e7edbae34ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:29.548788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6897bba-5374-474a-ace2-3e7edbae34ed admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:29.566824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:29.581020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1226/2454] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:29:29.581707 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f717765-e504-4b14-8c11-16da459d85a3 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.595218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f717765-e504-4b14-8c11-16da459d85a3 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.610327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934ba59-cb99-4ebc-a8a7-8d0a1cfed94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:29.612012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ba59-cb99-4ebc-a8a7-8d0a1cfed94b admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.612012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ba59-cb99-4ebc-a8a7-8d0a1cfed94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.612170 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ba59-cb99-4ebc-a8a7-8d0a1cfed94b admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.612829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934ba59-cb99-4ebc-a8a7-8d0a1cfed94b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:29.615524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934ba59-cb99-4ebc-a8a7-8d0a1cfed94b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:29.663924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6897bba-5374-474a-ace2-3e7edbae34ed admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.673242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1229/2455] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:29 2022] POST /identity/v3/projects => generated 393 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:29.680239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6897bba-5374-474a-ace2-3e7edbae34ed admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.785279 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0f0e1-eed1-459a-a95a-90122e2ee9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:29.787345 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0f0e1-eed1-459a-a95a-90122e2ee9e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.787866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0f0e1-eed1-459a-a95a-90122e2ee9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.788285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0f0e1-eed1-459a-a95a-90122e2ee9e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.789298 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0f0e1-eed1-459a-a95a-90122e2ee9e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:29.792735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0f0e1-eed1-459a-a95a-90122e2ee9e6 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:29.840729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934ba59-cb99-4ebc-a8a7-8d0a1cfed94b admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.845123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1227/2456] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:29 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:29.848520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0f0e1-eed1-459a-a95a-90122e2ee9e6 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.853286 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6934ba59-cb99-4ebc-a8a7-8d0a1cfed94b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:29.872982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0f0e1-eed1-459a-a95a-90122e2ee9e6 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.879412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29ac95-dc8b-4319-9fd5-741e08ec3c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:29.880307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29ac95-dc8b-4319-9fd5-741e08ec3c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.880706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29ac95-dc8b-4319-9fd5-741e08ec3c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.881157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29ac95-dc8b-4319-9fd5-741e08ec3c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.882084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29ac95-dc8b-4319-9fd5-741e08ec3c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:29.884988 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29ac95-dc8b-4319-9fd5-741e08ec3c2b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:29.892648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934ba59-cb99-4ebc-a8a7-8d0a1cfed94b admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.898336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:29.899404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.899763 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.900297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1228/2457] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:29.903170 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] PATH_INFO: `/v3/projects/99d7d9f2de3f4292b04c3501a063a869` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.904837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99d7d9f2de3f4292b04c3501a063a869)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:29.906196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29ac95-dc8b-4319-9fd5-741e08ec3c2b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.907032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:29.921166 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29ac95-dc8b-4319-9fd5-741e08ec3c2b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.926637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b089ad2-2371-4db0-8a4b-c349536696fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:29.927364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b089ad2-2371-4db0-8a4b-c349536696fb admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:29.927656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b089ad2-2371-4db0-8a4b-c349536696fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:29.927978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b089ad2-2371-4db0-8a4b-c349536696fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:29.928720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b089ad2-2371-4db0-8a4b-c349536696fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:29.930620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b089ad2-2371-4db0-8a4b-c349536696fb admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:29.946394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1229/2458] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:29.950980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b089ad2-2371-4db0-8a4b-c349536696fb admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:29.969376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b089ad2-2371-4db0-8a4b-c349536696fb admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:29.977337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99d7d9f2de3f4292b04c3501a063a869', 'request_id': 'req-a391ce3b-8086-490a-bf66-02ea715a4c94'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.987159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99d7d9f2de3f4292b04c3501a063a869', 'request_id': 'req-a391ce3b-8086-490a-bf66-02ea715a4c94'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:29.995523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99d7d9f2de3f4292b04c3501a063a869', 'request_id': 'req-a391ce3b-8086-490a-bf66-02ea715a4c94'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:30.014595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] The token cache is being invalidate because project 99d7d9f2de3f4292b04c3501a063a869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:30.015185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:30.015895 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a391ce3b-8086-490a-bf66-02ea715a4c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:30.017886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1230/2459] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:29 2022] DELETE /identity/v3/projects/99d7d9f2de3f4292b04c3501a063a869 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:30.023511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f44a1-d489-4b59-a88a-f55f83cff711 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.023792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f44a1-d489-4b59-a88a-f55f83cff711 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.024064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f44a1-d489-4b59-a88a-f55f83cff711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.053872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59852f12-fb28-41be-b6fa-988a9a487eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.054903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852f12-fb28-41be-b6fa-988a9a487eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.055280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-734f44a1-d489-4b59-a88a-f55f83cff711 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:30.055387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852f12-fb28-41be-b6fa-988a9a487eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.055764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852f12-fb28-41be-b6fa-988a9a487eb0 admin admin] PATH_INFO: `/v3/projects/da4feb4c4ad04ad4a3fd031e4a78618e/users/75bb30ebbcc14ce48b9fe0876dd8f3b4/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.056372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-734f44a1-d489-4b59-a88a-f55f83cff711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:30.065930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1231/2460] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:30.074743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97817632-9cc4-46b1-9f44-80ac89795d8a None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.075027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97817632-9cc4-46b1-9f44-80ac89795d8a None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.075326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97817632-9cc4-46b1-9f44-80ac89795d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.076682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59852f12-fb28-41be-b6fa-988a9a487eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da4feb4c4ad04ad4a3fd031e4a78618e, user_id=75bb30ebbcc14ce48b9fe0876dd8f3b4, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.079636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59852f12-fb28-41be-b6fa-988a9a487eb0 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.096215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1230/2461] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:29 2022] PUT /identity/v3/projects/da4feb4c4ad04ad4a3fd031e4a78618e/users/75bb30ebbcc14ce48b9fe0876dd8f3b4/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:30.096839 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-97817632-9cc4-46b1-9f44-80ac89795d8a None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:30.098092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-97817632-9cc4-46b1-9f44-80ac89795d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:30.107338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3c4d5-0f10-4b50-873f-eab6a665fcf4 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.107936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3c4d5-0f10-4b50-873f-eab6a665fcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.108460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3c4d5-0f10-4b50-873f-eab6a665fcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.113721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:30.140296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-53a3c4d5-0f10-4b50-873f-eab6a665fcf4 None None] MFA Rules not processed for user `75bb30ebbcc14ce48b9fe0876dd8f3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:30.141528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-53a3c4d5-0f10-4b50-873f-eab6a665fcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:30.186884 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1232/2462] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:30.187887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97817632-9cc4-46b1-9f44-80ac89795d8a None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.189194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:30.199208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97817632-9cc4-46b1-9f44-80ac89795d8a None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.205166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c49a88-c89c-4d6f-bfc0-3e18bf3ee4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.205842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c49a88-c89c-4d6f-bfc0-3e18bf3ee4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.206110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c49a88-c89c-4d6f-bfc0-3e18bf3ee4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.206390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c49a88-c89c-4d6f-bfc0-3e18bf3ee4f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.207646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c49a88-c89c-4d6f-bfc0-3e18bf3ee4f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:30.207953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c49a88-c89c-4d6f-bfc0-3e18bf3ee4f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.210148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c49a88-c89c-4d6f-bfc0-3e18bf3ee4f3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.223285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1233/2463] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:29:30.227357 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c49a88-c89c-4d6f-bfc0-3e18bf3ee4f3 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.228184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1231/2464] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:30.232620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3c4d5-0f10-4b50-873f-eab6a665fcf4 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.234027 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-53a3c4d5-0f10-4b50-873f-eab6a665fcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:30.238119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c49a88-c89c-4d6f-bfc0-3e18bf3ee4f3 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.242719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b52b6-f0a8-48e2-95ae-702fa3e08220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.243382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b52b6-f0a8-48e2-95ae-702fa3e08220 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.243662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b52b6-f0a8-48e2-95ae-702fa3e08220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.243937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b52b6-f0a8-48e2-95ae-702fa3e08220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.244627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b52b6-f0a8-48e2-95ae-702fa3e08220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.246679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b52b6-f0a8-48e2-95ae-702fa3e08220 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.248944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3c4d5-0f10-4b50-873f-eab6a665fcf4 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.254983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca052696-a426-46b7-87b5-46f0469c8fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.255688 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca052696-a426-46b7-87b5-46f0469c8fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.255969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca052696-a426-46b7-87b5-46f0469c8fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.256249 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca052696-a426-46b7-87b5-46f0469c8fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.256939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca052696-a426-46b7-87b5-46f0469c8fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.258845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca052696-a426-46b7-87b5-46f0469c8fa3 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.276405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1234/2465] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:30.280946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b52b6-f0a8-48e2-95ae-702fa3e08220 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.296949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1232/2466] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/projects => generated 393 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:30.298004 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca052696-a426-46b7-87b5-46f0469c8fa3 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.360368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b52b6-f0a8-48e2-95ae-702fa3e08220 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.375584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca052696-a426-46b7-87b5-46f0469c8fa3 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.382874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f421be-1f92-4032-88fb-9be9e2f1e116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.383677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f421be-1f92-4032-88fb-9be9e2f1e116 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.383957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f421be-1f92-4032-88fb-9be9e2f1e116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.384228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f421be-1f92-4032-88fb-9be9e2f1e116 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.385209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f421be-1f92-4032-88fb-9be9e2f1e116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.388181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f421be-1f92-4032-88fb-9be9e2f1e116 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.411454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1233/2467] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/users => generated 303 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:30.415463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f421be-1f92-4032-88fb-9be9e2f1e116 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.423837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a08d10-0bd5-45b1-8fc1-efb08e07ec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.424617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a08d10-0bd5-45b1-8fc1-efb08e07ec08 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.424968 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a08d10-0bd5-45b1-8fc1-efb08e07ec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.425312 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a08d10-0bd5-45b1-8fc1-efb08e07ec08 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.426075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a08d10-0bd5-45b1-8fc1-efb08e07ec08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.426165 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f421be-1f92-4032-88fb-9be9e2f1e116 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.428256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a08d10-0bd5-45b1-8fc1-efb08e07ec08 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.431746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dcabb-0b83-461b-8b38-c15f493dcf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.433321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dcabb-0b83-461b-8b38-c15f493dcf7d admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.433605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dcabb-0b83-461b-8b38-c15f493dcf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.433876 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dcabb-0b83-461b-8b38-c15f493dcf7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.434612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62dcabb-0b83-461b-8b38-c15f493dcf7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.436691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62dcabb-0b83-461b-8b38-c15f493dcf7d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.448957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1234/2468] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:30.454838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dcabb-0b83-461b-8b38-c15f493dcf7d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.458242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1235/2469] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:30.463239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a08d10-0bd5-45b1-8fc1-efb08e07ec08 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.469921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dcabb-0b83-461b-8b38-c15f493dcf7d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.473754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a08d10-0bd5-45b1-8fc1-efb08e07ec08 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.474830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1154d2-636d-469b-94bd-e787753c02e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.478631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286584fd-cf0c-4bfb-9f7f-ece8d079c1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.479345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286584fd-cf0c-4bfb-9f7f-ece8d079c1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.479668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286584fd-cf0c-4bfb-9f7f-ece8d079c1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.479989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286584fd-cf0c-4bfb-9f7f-ece8d079c1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.480400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1154d2-636d-469b-94bd-e787753c02e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.480699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1154d2-636d-469b-94bd-e787753c02e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.480804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286584fd-cf0c-4bfb-9f7f-ece8d079c1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.482317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1154d2-636d-469b-94bd-e787753c02e5 admin admin] PATH_INFO: `/v3/projects/eb531d64b84347c098dac87640bfee74/users/5e8d26c6fe034727ba1858b3064c7f5c/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.482444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286584fd-cf0c-4bfb-9f7f-ece8d079c1f6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.497236 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1236/2470] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:30.498218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286584fd-cf0c-4bfb-9f7f-ece8d079c1f6 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.508087 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1154d2-636d-469b-94bd-e787753c02e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb531d64b84347c098dac87640bfee74, user_id=5e8d26c6fe034727ba1858b3064c7f5c, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.510830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1154d2-636d-469b-94bd-e787753c02e5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.514733 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286584fd-cf0c-4bfb-9f7f-ece8d079c1f6 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.525626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031838dd-de5c-44a1-9d52-5480f512be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.526580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031838dd-de5c-44a1-9d52-5480f512be25 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.527001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031838dd-de5c-44a1-9d52-5480f512be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.527661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031838dd-de5c-44a1-9d52-5480f512be25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.528997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031838dd-de5c-44a1-9d52-5480f512be25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.530957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031838dd-de5c-44a1-9d52-5480f512be25 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.532638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:30.543390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1237/2471] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:30.546707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1235/2472] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:30 2022] PUT /identity/v3/projects/eb531d64b84347c098dac87640bfee74/users/5e8d26c6fe034727ba1858b3064c7f5c/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:30.547238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031838dd-de5c-44a1-9d52-5480f512be25 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.550502 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1154d2-636d-469b-94bd-e787753c02e5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.559770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031838dd-de5c-44a1-9d52-5480f512be25 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.623165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa26225-718f-4eef-ba7d-9188c1acc4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.624351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26225-718f-4eef-ba7d-9188c1acc4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.624730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26225-718f-4eef-ba7d-9188c1acc4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.625103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26225-718f-4eef-ba7d-9188c1acc4c3 admin admin] PATH_INFO: `/v3/projects/d4767021b5ca4e78a01e55280fced204/users/e387d72935434123a0b5dd6c9986711f/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.636981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1154d2-636d-469b-94bd-e787753c02e5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.646785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa26225-718f-4eef-ba7d-9188c1acc4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4767021b5ca4e78a01e55280fced204, user_id=e387d72935434123a0b5dd6c9986711f, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.648223 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3969c-7efd-4662-a838-5d271b218677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.648925 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3969c-7efd-4662-a838-5d271b218677 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.649206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3969c-7efd-4662-a838-5d271b218677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.649477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3969c-7efd-4662-a838-5d271b218677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.649568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa26225-718f-4eef-ba7d-9188c1acc4c3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.650206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3969c-7efd-4662-a838-5d271b218677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.651927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3969c-7efd-4662-a838-5d271b218677 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.668477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1236/2473] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:30.672279 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3969c-7efd-4662-a838-5d271b218677 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.672741 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1238/2474] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:30 2022] PUT /identity/v3/projects/d4767021b5ca4e78a01e55280fced204/users/e387d72935434123a0b5dd6c9986711f/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:30.684394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e19661-4805-45c3-a54d-f92335193f03 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.684775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e19661-4805-45c3-a54d-f92335193f03 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.685282 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e19661-4805-45c3-a54d-f92335193f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.686425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3969c-7efd-4662-a838-5d271b218677 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.693463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:30.720399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c3e19661-4805-45c3-a54d-f92335193f03 None None] MFA Rules not processed for user `e387d72935434123a0b5dd6c9986711f`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:30.721720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c3e19661-4805-45c3-a54d-f92335193f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:30.752838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45513169-4c18-4594-93ab-f1494ec5f631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.754005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45513169-4c18-4594-93ab-f1494ec5f631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.754291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45513169-4c18-4594-93ab-f1494ec5f631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.754560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45513169-4c18-4594-93ab-f1494ec5f631 admin admin] PATH_INFO: `/v3/projects/eb531d64b84347c098dac87640bfee74/users/5e8d26c6fe034727ba1858b3064c7f5c/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.756713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45513169-4c18-4594-93ab-f1494ec5f631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb531d64b84347c098dac87640bfee74, user_id=5e8d26c6fe034727ba1858b3064c7f5c, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.759597 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45513169-4c18-4594-93ab-f1494ec5f631 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.772089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1237/2475] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:30 2022] PUT /identity/v3/projects/eb531d64b84347c098dac87640bfee74/users/5e8d26c6fe034727ba1858b3064c7f5c/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:30.783192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e665e89-9d40-4578-8ccd-6319ee779b3f None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.783624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e665e89-9d40-4578-8ccd-6319ee779b3f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.783905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e665e89-9d40-4578-8ccd-6319ee779b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.797232 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1239/2476] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:30.801825 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e19661-4805-45c3-a54d-f92335193f03 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.812931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e19661-4805-45c3-a54d-f92335193f03 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:30.816927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-8e665e89-9d40-4578-8ccd-6319ee779b3f None None] MFA Rules not processed for user `5e8d26c6fe034727ba1858b3064c7f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:30.818335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8e665e89-9d40-4578-8ccd-6319ee779b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:30.880731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786c9a5-dfad-4d7a-a616-f5f4e9e20ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:30.881421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786c9a5-dfad-4d7a-a616-f5f4e9e20ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:30.881701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786c9a5-dfad-4d7a-a616-f5f4e9e20ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:30.882024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786c9a5-dfad-4d7a-a616-f5f4e9e20ad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:30.882721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786c9a5-dfad-4d7a-a616-f5f4e9e20ad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:30.884664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786c9a5-dfad-4d7a-a616-f5f4e9e20ad9 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:30.901627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1238/2477] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:30.906822 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:30.914108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e665e89-9d40-4578-8ccd-6319ee779b3f None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.915967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8e665e89-9d40-4578-8ccd-6319ee779b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:30.931634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1240/2478] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:30.938027 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786c9a5-dfad-4d7a-a616-f5f4e9e20ad9 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:30.986375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e665e89-9d40-4578-8ccd-6319ee779b3f None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:31.030856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:31.033624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786c9a5-dfad-4d7a-a616-f5f4e9e20ad9 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:31.043049 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50aac2-f41a-469b-812b-ab40b8ded793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:31.043882 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50aac2-f41a-469b-812b-ab40b8ded793 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.044380 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50aac2-f41a-469b-812b-ab40b8ded793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.044674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50aac2-f41a-469b-812b-ab40b8ded793 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.045242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50aac2-f41a-469b-812b-ab40b8ded793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:31.047348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50aac2-f41a-469b-812b-ab40b8ded793 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:31.048612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232764d-540e-45c7-9e83-7677f87fe746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:31.049491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232764d-540e-45c7-9e83-7677f87fe746 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.049950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232764d-540e-45c7-9e83-7677f87fe746 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.050423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232764d-540e-45c7-9e83-7677f87fe746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.076890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1241/2479] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:30 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:31.078552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50aac2-f41a-469b-812b-ab40b8ded793 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:31.082051 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3f50aac2-f41a-469b-812b-ab40b8ded793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:31.103146 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50aac2-f41a-469b-812b-ab40b8ded793 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:31.112424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:31.114614 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.115101 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.115994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] PATH_INFO: `/v3/users/22eeb910338e4e5f8f761a1ff2557c5a` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.117674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] RBAC: Authorizing `identity:delete_user(user_id=22eeb910338e4e5f8f761a1ff2557c5a)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:31.120041 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:31.168898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232764d-540e-45c7-9e83-7677f87fe746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:31.171054 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232764d-540e-45c7-9e83-7677f87fe746 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:31.186826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1239/2480] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 06:29:31.188392 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232764d-540e-45c7-9e83-7677f87fe746 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:31.199083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22eeb910338e4e5f8f761a1ff2557c5a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:31.214979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22eeb910338e4e5f8f761a1ff2557c5a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:31.216327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22eeb910338e4e5f8f761a1ff2557c5a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:31.218966 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232764d-540e-45c7-9e83-7677f87fe746 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:31.229439 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979ba0f-26b7-4656-8ae3-85fbf12010e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:31.230277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979ba0f-26b7-4656-8ae3-85fbf12010e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.230638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979ba0f-26b7-4656-8ae3-85fbf12010e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.231185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979ba0f-26b7-4656-8ae3-85fbf12010e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.232046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979ba0f-26b7-4656-8ae3-85fbf12010e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:31.234279 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979ba0f-26b7-4656-8ae3-85fbf12010e7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:31.239422 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22eeb910338e4e5f8f761a1ff2557c5a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:31.251886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1240/2481] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:31.256845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979ba0f-26b7-4656-8ae3-85fbf12010e7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:31.266210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22eeb910338e4e5f8f761a1ff2557c5a'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:31.271104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979ba0f-26b7-4656-8ae3-85fbf12010e7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:31.274671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1242/2482] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:31 2022] DELETE /identity/v3/users/22eeb910338e4e5f8f761a1ff2557c5a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:31.275548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:31.277167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fe65c-a7d4-4f3c-8dce-682c016128a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:31.277846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fe65c-a7d4-4f3c-8dce-682c016128a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.278125 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fe65c-a7d4-4f3c-8dce-682c016128a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.278399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fe65c-a7d4-4f3c-8dce-682c016128a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.279124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fe65c-a7d4-4f3c-8dce-682c016128a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:31.280598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:31.280771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fe65c-a7d4-4f3c-8dce-682c016128a4 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:31.294789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1241/2483] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:31.300907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830c654-3266-4cee-a643-ab629e744f2f None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.301827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830c654-3266-4cee-a643-ab629e744f2f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.302239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830c654-3266-4cee-a643-ab629e744f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.323702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2830c654-3266-4cee-a643-ab629e744f2f None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:31.325022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2830c654-3266-4cee-a643-ab629e744f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:31.365620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb97b0-e21c-4841-aed7-4ba63c82835d admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:31.374019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b0fd0-6e76-42a7-b8fc-2262ec33327b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:31.375021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b0fd0-6e76-42a7-b8fc-2262ec33327b service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.375629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b0fd0-6e76-42a7-b8fc-2262ec33327b service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.375975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b0fd0-6e76-42a7-b8fc-2262ec33327b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.378163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b44b0fd0-6e76-42a7-b8fc-2262ec33327b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:31.389771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:31.416376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1242/2484] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:31.417430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830c654-3266-4cee-a643-ab629e744f2f None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:31.419019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2830c654-3266-4cee-a643-ab629e744f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:31.437185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830c654-3266-4cee-a643-ab629e744f2f None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:31.443916 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe83906-ce6b-4491-be0f-a9199b1eaab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:31.445375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83906-ce6b-4491-be0f-a9199b1eaab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.445718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83906-ce6b-4491-be0f-a9199b1eaab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.446033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83906-ce6b-4491-be0f-a9199b1eaab4 admin admin] PATH_INFO: `/v3/projects/7d6ea27e5f7448c1a159a72de27cd47d/users/06dae7a75934405fbafedaa6d60655d8/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.462598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe83906-ce6b-4491-be0f-a9199b1eaab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d6ea27e5f7448c1a159a72de27cd47d, user_id=06dae7a75934405fbafedaa6d60655d8, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:31.465356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe83906-ce6b-4491-be0f-a9199b1eaab4 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:31.471083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b0fd0-6e76-42a7-b8fc-2262ec33327b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:31.472715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b0fd0-6e76-42a7-b8fc-2262ec33327b service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:31.482788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1243/2485] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:31 2022] PUT /identity/v3/projects/7d6ea27e5f7448c1a159a72de27cd47d/users/06dae7a75934405fbafedaa6d60655d8/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:31.484136 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe83906-ce6b-4491-be0f-a9199b1eaab4 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:31.485890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4fe83906-ce6b-4491-be0f-a9199b1eaab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:31.494499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1243/2486] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:31.495939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc695329-c47c-4f6e-96e9-09e4d434f813 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.496402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc695329-c47c-4f6e-96e9-09e4d434f813 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.496685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc695329-c47c-4f6e-96e9-09e4d434f813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.543893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-cc695329-c47c-4f6e-96e9-09e4d434f813 None None] MFA Rules not processed for user `06dae7a75934405fbafedaa6d60655d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:31.546565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cc695329-c47c-4f6e-96e9-09e4d434f813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:31.553331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:31.563859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe83906-ce6b-4491-be0f-a9199b1eaab4 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:31.573427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82192635-01b7-4deb-a07c-4907c106be0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:31.574127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82192635-01b7-4deb-a07c-4907c106be0f service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.574585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82192635-01b7-4deb-a07c-4907c106be0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.574772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82192635-01b7-4deb-a07c-4907c106be0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.678513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1244/2487] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:31.683222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc695329-c47c-4f6e-96e9-09e4d434f813 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:31.701920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82192635-01b7-4deb-a07c-4907c106be0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:31.705119 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82192635-01b7-4deb-a07c-4907c106be0f service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:31.724191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1244/2488] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:31.815842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc695329-c47c-4f6e-96e9-09e4d434f813 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:31.824347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233ce4f-9390-441e-a5ed-f3d02ebb8f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:31.824591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233ce4f-9390-441e-a5ed-f3d02ebb8f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.824898 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233ce4f-9390-441e-a5ed-f3d02ebb8f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.825191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233ce4f-9390-441e-a5ed-f3d02ebb8f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.826015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233ce4f-9390-441e-a5ed-f3d02ebb8f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:31.828551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233ce4f-9390-441e-a5ed-f3d02ebb8f7f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:31.875947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1245/2489] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:31 2022] POST /identity/v3/projects => generated 379 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:31.880417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82192635-01b7-4deb-a07c-4907c106be0f service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:31.890958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82192635-01b7-4deb-a07c-4907c106be0f service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:31.987900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861877f-bc74-4f79-96ca-3d40f5149d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:31.988868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861877f-bc74-4f79-96ca-3d40f5149d64 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:31.989156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861877f-bc74-4f79-96ca-3d40f5149d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:31.989432 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861877f-bc74-4f79-96ca-3d40f5149d64 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:31.990229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861877f-bc74-4f79-96ca-3d40f5149d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:31.992828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861877f-bc74-4f79-96ca-3d40f5149d64 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:32.055978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1245/2490] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:31 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:32.060268 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233ce4f-9390-441e-a5ed-f3d02ebb8f7f admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:32.079888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233ce4f-9390-441e-a5ed-f3d02ebb8f7f admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:32.085927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfef00-0442-4527-9f85-6432e5fb8e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:32.086776 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfef00-0442-4527-9f85-6432e5fb8e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:32.087062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfef00-0442-4527-9f85-6432e5fb8e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:32.087377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfef00-0442-4527-9f85-6432e5fb8e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:32.088116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfef00-0442-4527-9f85-6432e5fb8e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:32.090334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfef00-0442-4527-9f85-6432e5fb8e0b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:32.103196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1246/2491] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:32.108046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861877f-bc74-4f79-96ca-3d40f5149d64 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:32.119351 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861877f-bc74-4f79-96ca-3d40f5149d64 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:32.124710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8de078-5c5f-46f7-8ffb-f6516e70e271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:32.125734 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8de078-5c5f-46f7-8ffb-f6516e70e271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:32.126086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8de078-5c5f-46f7-8ffb-f6516e70e271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:32.126497 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8de078-5c5f-46f7-8ffb-f6516e70e271 admin admin] PATH_INFO: `/v3/projects/af9bf37173154fe186166a07464db4c8/users/741a75546d5a478ba4ace7f5d2da34d9/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:32.143854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8de078-5c5f-46f7-8ffb-f6516e70e271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9bf37173154fe186166a07464db4c8, user_id=741a75546d5a478ba4ace7f5d2da34d9, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:32.146673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8de078-5c5f-46f7-8ffb-f6516e70e271 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:32.173619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1246/2492] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:32 2022] PUT /identity/v3/projects/af9bf37173154fe186166a07464db4c8/users/741a75546d5a478ba4ace7f5d2da34d9/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:32.179353 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfef00-0442-4527-9f85-6432e5fb8e0b admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:32.191411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfef00-0442-4527-9f85-6432e5fb8e0b admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:32.203089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:32.283415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf060493-a2a6-42d3-a71d-edeb94f29c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:32.284103 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060493-a2a6-42d3-a71d-edeb94f29c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:32.284523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060493-a2a6-42d3-a71d-edeb94f29c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:32.284615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060493-a2a6-42d3-a71d-edeb94f29c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:32.285359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf060493-a2a6-42d3-a71d-edeb94f29c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:32.287396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf060493-a2a6-42d3-a71d-edeb94f29c05 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:32.301245 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1247/2493] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:32.305546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8de078-5c5f-46f7-8ffb-f6516e70e271 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:32.318806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8de078-5c5f-46f7-8ffb-f6516e70e271 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:32.326577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dc3e6-c396-46ed-a513-687b1ad51ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:32.327962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dc3e6-c396-46ed-a513-687b1ad51ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:32.328314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dc3e6-c396-46ed-a513-687b1ad51ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:32.328673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dc3e6-c396-46ed-a513-687b1ad51ad7 admin admin] PATH_INFO: `/v3/projects/af9bf37173154fe186166a07464db4c8/users/741a75546d5a478ba4ace7f5d2da34d9/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:32.331332 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dc3e6-c396-46ed-a513-687b1ad51ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9bf37173154fe186166a07464db4c8, user_id=741a75546d5a478ba4ace7f5d2da34d9, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:32.334550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dc3e6-c396-46ed-a513-687b1ad51ad7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:32.344242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf060493-a2a6-42d3-a71d-edeb94f29c05 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:32.348916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cf060493-a2a6-42d3-a71d-edeb94f29c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:32.356656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1247/2494] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:32 2022] PUT /identity/v3/projects/af9bf37173154fe186166a07464db4c8/users/741a75546d5a478ba4ace7f5d2da34d9/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:32.369277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49255b3e-2667-42bb-a7f5-7ad94a21e8d0 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:32.369897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49255b3e-2667-42bb-a7f5-7ad94a21e8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:32.370617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49255b3e-2667-42bb-a7f5-7ad94a21e8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:32.370766 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf060493-a2a6-42d3-a71d-edeb94f29c05 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:32.424303 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-49255b3e-2667-42bb-a7f5-7ad94a21e8d0 None None] MFA Rules not processed for user `741a75546d5a478ba4ace7f5d2da34d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:32.425674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-49255b3e-2667-42bb-a7f5-7ad94a21e8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:32.446071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:32.446983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:32.447290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:32.447554 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] PATH_INFO: `/v3/projects/26ae257e47dc48b69a5f81d22e9733e0` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:32.449481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26ae257e47dc48b69a5f81d22e9733e0)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:32.451173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:32.488074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:32.536021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26ae257e47dc48b69a5f81d22e9733e0', 'request_id': 'req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:32.546362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26ae257e47dc48b69a5f81d22e9733e0', 'request_id': 'req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:32.558283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26ae257e47dc48b69a5f81d22e9733e0', 'request_id': 'req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:32.558791 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1248/2495] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:32.567310 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49255b3e-2667-42bb-a7f5-7ad94a21e8d0 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:32.574911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] The token cache is being invalidate because project 26ae257e47dc48b69a5f81d22e9733e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:32.577714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:32.578532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:32.580758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1248/2496] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:32 2022] DELETE /identity/v3/projects/26ae257e47dc48b69a5f81d22e9733e0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:32.588760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:32.590944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:32.674634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49255b3e-2667-42bb-a7f5-7ad94a21e8d0 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:32.678286 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:32.713419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc323f4-3a5d-42ea-960c-3cdcf2db2f53 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:32.719095 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:32.720016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:32.720290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:32.720637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] PATH_INFO: `/v3/users/f1709f21864e47dc8c3a2367f1789299` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:32.721787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1709f21864e47dc8c3a2367f1789299)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:32.723923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:32.766093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb68e66-191f-459a-acf3-2f76c9d75554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:32.766777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb68e66-191f-459a-acf3-2f76c9d75554 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:32.767096 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb68e66-191f-459a-acf3-2f76c9d75554 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:32.767358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb68e66-191f-459a-acf3-2f76c9d75554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:32.769514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-bbb68e66-191f-459a-acf3-2f76c9d75554 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:32.813826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1709f21864e47dc8c3a2367f1789299'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:32.832395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1709f21864e47dc8c3a2367f1789299'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:32.832395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1709f21864e47dc8c3a2367f1789299'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:32.844855 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1709f21864e47dc8c3a2367f1789299'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:32.859234 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1709f21864e47dc8c3a2367f1789299'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:32.866811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1249/2497] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:32 2022] DELETE /identity/v3/users/f1709f21864e47dc8c3a2367f1789299 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:32.877093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb68e66-191f-459a-acf3-2f76c9d75554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:32.878908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb68e66-191f-459a-acf3-2f76c9d75554 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:32.891356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1249/2498] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:32.954565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:32.956144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:33.090420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bf98b-4757-4cff-b646-4999afc357b0 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:33.096071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458537c8-d187-4bea-979d-86a513717d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:33.096905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458537c8-d187-4bea-979d-86a513717d96 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:33.097192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458537c8-d187-4bea-979d-86a513717d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:33.097467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458537c8-d187-4bea-979d-86a513717d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:33.099569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-458537c8-d187-4bea-979d-86a513717d96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:33.193651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458537c8-d187-4bea-979d-86a513717d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:33.195619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458537c8-d187-4bea-979d-86a513717d96 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:33.214437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1250/2499] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:33.326924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb68e66-191f-459a-acf3-2f76c9d75554 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:33.359331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb68e66-191f-459a-acf3-2f76c9d75554 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:33.422451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:33.444427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458537c8-d187-4bea-979d-86a513717d96 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:33.445866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-458537c8-d187-4bea-979d-86a513717d96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:33.460341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:33.461605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:33.461901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:33.462181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] PATH_INFO: `/v3/projects/9eb96babd12848e8bd45ee7c232dc9cc` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:33.463970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb96babd12848e8bd45ee7c232dc9cc)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:33.466058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:33.468885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:33.556494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458537c8-d187-4bea-979d-86a513717d96 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:33.584338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb96babd12848e8bd45ee7c232dc9cc', 'request_id': 'req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:33.597534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb96babd12848e8bd45ee7c232dc9cc', 'request_id': 'req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:33.606383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb96babd12848e8bd45ee7c232dc9cc', 'request_id': 'req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:33.630463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] The token cache is being invalidate because project 9eb96babd12848e8bd45ee7c232dc9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:33.630827 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:33.631585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:33.633637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1250/2500] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:33 2022] DELETE /identity/v3/projects/9eb96babd12848e8bd45ee7c232dc9cc => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:33.635236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:33.642150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:33.657181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6733347-e474-40b4-bd5f-0dc03d2e0eec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:33.658010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733347-e474-40b4-bd5f-0dc03d2e0eec service placement] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:33.658402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733347-e474-40b4-bd5f-0dc03d2e0eec service placement] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:33.658553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733347-e474-40b4-bd5f-0dc03d2e0eec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:33.660779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a6733347-e474-40b4-bd5f-0dc03d2e0eec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:33.675452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6733347-e474-40b4-bd5f-0dc03d2e0eec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:33.677500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6733347-e474-40b4-bd5f-0dc03d2e0eec service placement] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:33.690388 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1251/2501] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:33.691739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c953ce-707d-48b8-a430-2ed219e21f84 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:33.693603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c953ce-707d-48b8-a430-2ed219e21f84 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:33.693603 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c953ce-707d-48b8-a430-2ed219e21f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:33.723033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-e5c953ce-707d-48b8-a430-2ed219e21f84 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:33.724583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e5c953ce-707d-48b8-a430-2ed219e21f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:33.769448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e50a3-d24c-4884-ad8c-0324d5531ddd admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:33.778938 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:33.780304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:33.780831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:33.781185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] PATH_INFO: `/v3/users/93ae888c6e364e2e96ff52e63415a0e5` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:33.782994 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ae888c6e364e2e96ff52e63415a0e5)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:33.785575 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:33.805239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1252/2502] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:33.817953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b5030-883f-49cc-a9bf-1b45a1aed770 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:33.817953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b5030-883f-49cc-a9bf-1b45a1aed770 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:33.817953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b5030-883f-49cc-a9bf-1b45a1aed770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:33.838008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-bf1b5030-883f-49cc-a9bf-1b45a1aed770 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:33.839310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-bf1b5030-883f-49cc-a9bf-1b45a1aed770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:33.853333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1253/2503] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:33.854424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b5030-883f-49cc-a9bf-1b45a1aed770 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:33.868048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b5030-883f-49cc-a9bf-1b45a1aed770 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:33.875718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e19406-9195-4bf8-b591-a4abeb1ca223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:33.877395 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e19406-9195-4bf8-b591-a4abeb1ca223 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:33.877714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e19406-9195-4bf8-b591-a4abeb1ca223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:33.877906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e19406-9195-4bf8-b591-a4abeb1ca223 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:33.879029 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e19406-9195-4bf8-b591-a4abeb1ca223 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:33.879372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e19406-9195-4bf8-b591-a4abeb1ca223 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:33.881056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e19406-9195-4bf8-b591-a4abeb1ca223 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:33.883760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ae888c6e364e2e96ff52e63415a0e5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:33.892842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ae888c6e364e2e96ff52e63415a0e5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:33.904958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1254/2504] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:29:33.907182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ae888c6e364e2e96ff52e63415a0e5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:33.907182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ae888c6e364e2e96ff52e63415a0e5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:33.907182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ae888c6e364e2e96ff52e63415a0e5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:33.908642 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e19406-9195-4bf8-b591-a4abeb1ca223 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:33.913500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-07e19406-9195-4bf8-b591-a4abeb1ca223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:33.926464 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:33.931030 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e19406-9195-4bf8-b591-a4abeb1ca223 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:33.940392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abed1b8-2e08-4805-ab40-7b7429554e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:33.941166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abed1b8-2e08-4805-ab40-7b7429554e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:33.941519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abed1b8-2e08-4805-ab40-7b7429554e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:33.942463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abed1b8-2e08-4805-ab40-7b7429554e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:33.943371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abed1b8-2e08-4805-ab40-7b7429554e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:33.948108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1251/2505] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:33 2022] DELETE /identity/v3/users/93ae888c6e364e2e96ff52e63415a0e5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:33.948559 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abed1b8-2e08-4805-ab40-7b7429554e69 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.003300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1255/2506] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:33 2022] POST /identity/v3/projects => generated 389 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:34.005694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.090325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f962-3f79-43a9-8cf5-375755f6fa90 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.176377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abed1b8-2e08-4805-ab40-7b7429554e69 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.178467 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3abed1b8-2e08-4805-ab40-7b7429554e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:34.182144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a81d4-dd08-4d57-8d17-5c4f0b2d69b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.183174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a81d4-dd08-4d57-8d17-5c4f0b2d69b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.183692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a81d4-dd08-4d57-8d17-5c4f0b2d69b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.184130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a81d4-dd08-4d57-8d17-5c4f0b2d69b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.184888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a81d4-dd08-4d57-8d17-5c4f0b2d69b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.186905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a81d4-dd08-4d57-8d17-5c4f0b2d69b1 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.198562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:34.205620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abed1b8-2e08-4805-ab40-7b7429554e69 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.213439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.214806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.217249 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.217544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] PATH_INFO: `/v3/projects/27aa4fc6d2bd46d3a3492cf0b1d7ae9d` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.218975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27aa4fc6d2bd46d3a3492cf0b1d7ae9d)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.222434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1252/2507] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/users => generated 295 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:34.222570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.226861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a81d4-dd08-4d57-8d17-5c4f0b2d69b1 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.237412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a81d4-dd08-4d57-8d17-5c4f0b2d69b1 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.242659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942094ea-91fe-4220-ac98-6d74ee320db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.243720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942094ea-91fe-4220-ac98-6d74ee320db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.244140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942094ea-91fe-4220-ac98-6d74ee320db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.244986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942094ea-91fe-4220-ac98-6d74ee320db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.245425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942094ea-91fe-4220-ac98-6d74ee320db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.247053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942094ea-91fe-4220-ac98-6d74ee320db5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.261924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1253/2508] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:34.267545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942094ea-91fe-4220-ac98-6d74ee320db5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.279912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942094ea-91fe-4220-ac98-6d74ee320db5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.282277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27aa4fc6d2bd46d3a3492cf0b1d7ae9d', 'request_id': 'req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:34.290231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27aa4fc6d2bd46d3a3492cf0b1d7ae9d', 'request_id': 'req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:34.297465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27aa4fc6d2bd46d3a3492cf0b1d7ae9d', 'request_id': 'req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:34.313222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] The token cache is being invalidate because project 27aa4fc6d2bd46d3a3492cf0b1d7ae9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:34.313517 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:34.314469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d88390b4-5bfb-4eaa-b91d-b21df522e4b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:34.316165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1256/2509] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:34 2022] DELETE /identity/v3/projects/27aa4fc6d2bd46d3a3492cf0b1d7ae9d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:34.323035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a2c8a-9821-4508-adb5-5c390502c5d2 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.323398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a2c8a-9821-4508-adb5-5c390502c5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.323684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a2c8a-9821-4508-adb5-5c390502c5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.346165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-549a2c8a-9821-4508-adb5-5c390502c5d2 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:34.347191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6069-4d7c-41a6-acf2-4cbff4c272e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.347375 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-549a2c8a-9821-4508-adb5-5c390502c5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:34.348349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6069-4d7c-41a6-acf2-4cbff4c272e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.348883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6069-4d7c-41a6-acf2-4cbff4c272e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.349251 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6069-4d7c-41a6-acf2-4cbff4c272e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.350910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6069-4d7c-41a6-acf2-4cbff4c272e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.353244 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6069-4d7c-41a6-acf2-4cbff4c272e8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.364425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1257/2510] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:34.368537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1254/2511] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:34.368675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e947d-4059-4157-924a-4764bcdc037c None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.368675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e947d-4059-4157-924a-4764bcdc037c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.368675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e947d-4059-4157-924a-4764bcdc037c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.369514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6069-4d7c-41a6-acf2-4cbff4c272e8 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.371221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-accc6069-4d7c-41a6-acf2-4cbff4c272e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:34.384956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6069-4d7c-41a6-acf2-4cbff4c272e8 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.389071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-533e947d-4059-4157-924a-4764bcdc037c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:34.389835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2468a-567d-43e0-a7aa-03ab180fb879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.390274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-533e947d-4059-4157-924a-4764bcdc037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:34.390736 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2468a-567d-43e0-a7aa-03ab180fb879 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.391021 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2468a-567d-43e0-a7aa-03ab180fb879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.391339 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2468a-567d-43e0-a7aa-03ab180fb879 admin admin] PATH_INFO: `/v3/projects/e9bae52f7f3649f0839ec9176ccadecf/users/2771829ad13b4bb69a855eb04825d513/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.398152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1258/2512] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:34.402553 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e947d-4059-4157-924a-4764bcdc037c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.403596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:34.409104 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2468a-567d-43e0-a7aa-03ab180fb879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9bae52f7f3649f0839ec9176ccadecf, user_id=2771829ad13b4bb69a855eb04825d513, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.411880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2468a-567d-43e0-a7aa-03ab180fb879 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.412676 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e947d-4059-4157-924a-4764bcdc037c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.417957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55789c-b37d-4457-8802-e157c62735ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.418615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55789c-b37d-4457-8802-e157c62735ff admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.418903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55789c-b37d-4457-8802-e157c62735ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.419175 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55789c-b37d-4457-8802-e157c62735ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.419951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55789c-b37d-4457-8802-e157c62735ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:34.420263 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55789c-b37d-4457-8802-e157c62735ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.421854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55789c-b37d-4457-8802-e157c62735ff admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.427834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1255/2513] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:34 2022] PUT /identity/v3/projects/e9bae52f7f3649f0839ec9176ccadecf/users/2771829ad13b4bb69a855eb04825d513/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:34.437305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1259/2514] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 06:29:34.437880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d57d7d-a4b1-426e-8b43-db9b9165d732 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.438176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d57d7d-a4b1-426e-8b43-db9b9165d732 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.438453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d57d7d-a4b1-426e-8b43-db9b9165d732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.441339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55789c-b37d-4457-8802-e157c62735ff admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.451259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55789c-b37d-4457-8802-e157c62735ff admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.464982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c1d57d7d-a4b1-426e-8b43-db9b9165d732 None None] MFA Rules not processed for user `2771829ad13b4bb69a855eb04825d513`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:34.466372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c1d57d7d-a4b1-426e-8b43-db9b9165d732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:34.514095 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13c169-216c-4f29-8398-25025fd0299f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.514776 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13c169-216c-4f29-8398-25025fd0299f admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.515060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13c169-216c-4f29-8398-25025fd0299f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.515368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13c169-216c-4f29-8398-25025fd0299f admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.516072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13c169-216c-4f29-8398-25025fd0299f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.518172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13c169-216c-4f29-8398-25025fd0299f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.540355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1256/2515] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:34.545101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d57d7d-a4b1-426e-8b43-db9b9165d732 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.549162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1260/2516] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/projects => generated 401 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:34.553407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13c169-216c-4f29-8398-25025fd0299f admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.559127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d57d7d-a4b1-426e-8b43-db9b9165d732 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.612480 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:34.629926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d66447-7488-4073-9061-f93814e0a67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.630757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d66447-7488-4073-9061-f93814e0a67c admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.631046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d66447-7488-4073-9061-f93814e0a67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.631380 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d66447-7488-4073-9061-f93814e0a67c admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.632662 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d66447-7488-4073-9061-f93814e0a67c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.632787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13c169-216c-4f29-8398-25025fd0299f admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.635313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d66447-7488-4073-9061-f93814e0a67c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.637648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e9dbf-2430-4d6a-8645-8d7c707c020c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.638773 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e9dbf-2430-4d6a-8645-8d7c707c020c admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.639171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e9dbf-2430-4d6a-8645-8d7c707c020c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.639724 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e9dbf-2430-4d6a-8645-8d7c707c020c admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.640911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e9dbf-2430-4d6a-8645-8d7c707c020c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.642884 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e9dbf-2430-4d6a-8645-8d7c707c020c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.666879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1257/2517] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:34.667181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1261/2518] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/users => generated 301 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:34.668369 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e9dbf-2430-4d6a-8645-8d7c707c020c admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.668477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d66447-7488-4073-9061-f93814e0a67c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.678238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e9dbf-2430-4d6a-8645-8d7c707c020c admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.731718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d66447-7488-4073-9061-f93814e0a67c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.736260 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548859fe-5eaf-41c2-b516-7c3289407445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.736423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef846a26-facd-4b16-930c-0b491a101d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.736967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548859fe-5eaf-41c2-b516-7c3289407445 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.737076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef846a26-facd-4b16-930c-0b491a101d00 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.737218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548859fe-5eaf-41c2-b516-7c3289407445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.737332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef846a26-facd-4b16-930c-0b491a101d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.737485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548859fe-5eaf-41c2-b516-7c3289407445 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.737640 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef846a26-facd-4b16-930c-0b491a101d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.738189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548859fe-5eaf-41c2-b516-7c3289407445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.738299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef846a26-facd-4b16-930c-0b491a101d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.739973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef846a26-facd-4b16-930c-0b491a101d00 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.740192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548859fe-5eaf-41c2-b516-7c3289407445 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.748766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:34.750557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1262/2519] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:34.755805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef846a26-facd-4b16-930c-0b491a101d00 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.763441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1258/2520] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/users => generated 301 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:34.766649 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef846a26-facd-4b16-930c-0b491a101d00 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.767151 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548859fe-5eaf-41c2-b516-7c3289407445 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.771868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc981df-6d03-42b0-bbd8-d46beccc74da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.772536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc981df-6d03-42b0-bbd8-d46beccc74da admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.772819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc981df-6d03-42b0-bbd8-d46beccc74da admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.773097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc981df-6d03-42b0-bbd8-d46beccc74da admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.773846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc981df-6d03-42b0-bbd8-d46beccc74da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.775572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc981df-6d03-42b0-bbd8-d46beccc74da admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.777517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548859fe-5eaf-41c2-b516-7c3289407445 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.783041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ce682-3d03-4220-9755-e6e41b3dd427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.783777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ce682-3d03-4220-9755-e6e41b3dd427 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.784061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ce682-3d03-4220-9755-e6e41b3dd427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.784324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ce682-3d03-4220-9755-e6e41b3dd427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.785070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ce682-3d03-4220-9755-e6e41b3dd427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.785956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1263/2521] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:34.786876 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ce682-3d03-4220-9755-e6e41b3dd427 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.789925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc981df-6d03-42b0-bbd8-d46beccc74da admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.797427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1259/2522] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:34.800302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc981df-6d03-42b0-bbd8-d46beccc74da admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.801360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ce682-3d03-4220-9755-e6e41b3dd427 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.805434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d44ae-834e-4ea9-8f54-b31e8c07e22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.806343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d44ae-834e-4ea9-8f54-b31e8c07e22c admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.806804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d44ae-834e-4ea9-8f54-b31e8c07e22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.807110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d44ae-834e-4ea9-8f54-b31e8c07e22c admin admin] PATH_INFO: `/v3/projects/d5f1a3e07ca24585bbb746b4a768d9cb/users/1bea5d0fe6d34482b85ac668fe33fb73/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.811840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ce682-3d03-4220-9755-e6e41b3dd427 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.816833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef6f24-828a-4cf0-adca-f4e47ddcb680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.817721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef6f24-828a-4cf0-adca-f4e47ddcb680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.818008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef6f24-828a-4cf0-adca-f4e47ddcb680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.818289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef6f24-828a-4cf0-adca-f4e47ddcb680 admin admin] PATH_INFO: `/v3/projects/047b352008824dbba7ce6b4474d9566b/users/da99b1840cb64000910238f45e82ef65/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.821761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d44ae-834e-4ea9-8f54-b31e8c07e22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5f1a3e07ca24585bbb746b4a768d9cb, user_id=1bea5d0fe6d34482b85ac668fe33fb73, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.824519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d44ae-834e-4ea9-8f54-b31e8c07e22c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.833412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef6f24-828a-4cf0-adca-f4e47ddcb680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=047b352008824dbba7ce6b4474d9566b, user_id=da99b1840cb64000910238f45e82ef65, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.836592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef6f24-828a-4cf0-adca-f4e47ddcb680 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.839119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1264/2523] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:34 2022] PUT /identity/v3/projects/d5f1a3e07ca24585bbb746b4a768d9cb/users/1bea5d0fe6d34482b85ac668fe33fb73/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:34.849527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0880-18cc-4d85-89a9-9123fc8a1cee None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.849816 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0880-18cc-4d85-89a9-9123fc8a1cee None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.850121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0880-18cc-4d85-89a9-9123fc8a1cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.850565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1260/2524] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:34 2022] PUT /identity/v3/projects/047b352008824dbba7ce6b4474d9566b/users/da99b1840cb64000910238f45e82ef65/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:34.854320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef6f24-828a-4cf0-adca-f4e47ddcb680 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.876916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-3efc0880-18cc-4d85-89a9-9123fc8a1cee None None] MFA Rules not processed for user `1bea5d0fe6d34482b85ac668fe33fb73`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:34.878064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3efc0880-18cc-4d85-89a9-9123fc8a1cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:34.915619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef6f24-828a-4cf0-adca-f4e47ddcb680 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.920422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b282cc5-8f6f-4f78-8af4-c7987bdbd686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.921100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b282cc5-8f6f-4f78-8af4-c7987bdbd686 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.921382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b282cc5-8f6f-4f78-8af4-c7987bdbd686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.921657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b282cc5-8f6f-4f78-8af4-c7987bdbd686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.922422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b282cc5-8f6f-4f78-8af4-c7987bdbd686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.924072 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b282cc5-8f6f-4f78-8af4-c7987bdbd686 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.935335 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1261/2525] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:34.940383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b282cc5-8f6f-4f78-8af4-c7987bdbd686 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.943325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1265/2526] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:34.947922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc0880-18cc-4d85-89a9-9123fc8a1cee None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:34.952269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b282cc5-8f6f-4f78-8af4-c7987bdbd686 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.957002 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27fb5e-498c-4fda-bada-57be0ae5c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.957897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27fb5e-498c-4fda-bada-57be0ae5c25b admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.958185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27fb5e-498c-4fda-bada-57be0ae5c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.958289 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc0880-18cc-4d85-89a9-9123fc8a1cee None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:34.958461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27fb5e-498c-4fda-bada-57be0ae5c25b admin admin] PATH_INFO: `/v3/projects/047b352008824dbba7ce6b4474d9566b/users/da99b1840cb64000910238f45e82ef65/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.960473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27fb5e-498c-4fda-bada-57be0ae5c25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=047b352008824dbba7ce6b4474d9566b, user_id=da99b1840cb64000910238f45e82ef65, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.962695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cc72c-c679-4165-9fb2-1235a7caad9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:34.963142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27fb5e-498c-4fda-bada-57be0ae5c25b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.963373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cc72c-c679-4165-9fb2-1235a7caad9f admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.963652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cc72c-c679-4165-9fb2-1235a7caad9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.963980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cc72c-c679-4165-9fb2-1235a7caad9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:34.964669 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cc72c-c679-4165-9fb2-1235a7caad9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:34.966540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cc72c-c679-4165-9fb2-1235a7caad9f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:34.974369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1262/2527] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:34 2022] PUT /identity/v3/projects/047b352008824dbba7ce6b4474d9566b/users/da99b1840cb64000910238f45e82ef65/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:34.985888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54ef3-062a-4f87-a745-de218965d3bb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:34.986175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54ef3-062a-4f87-a745-de218965d3bb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:34.986460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54ef3-062a-4f87-a745-de218965d3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.000448 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1266/2528] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:35.002362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cc72c-c679-4165-9fb2-1235a7caad9f admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.010577 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:35.016813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c4c54ef3-062a-4f87-a745-de218965d3bb None None] MFA Rules not processed for user `da99b1840cb64000910238f45e82ef65`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:35.018042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c4c54ef3-062a-4f87-a745-de218965d3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:35.084137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cc72c-c679-4165-9fb2-1235a7caad9f admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:35.095442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d10192-5031-42b7-8fc7-e3ab8fcbebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:35.098412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d10192-5031-42b7-8fc7-e3ab8fcbebf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.098717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d10192-5031-42b7-8fc7-e3ab8fcbebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.099086 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d10192-5031-42b7-8fc7-e3ab8fcbebf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.099722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d10192-5031-42b7-8fc7-e3ab8fcbebf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:35.101745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d10192-5031-42b7-8fc7-e3ab8fcbebf6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:35.137877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1263/2529] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:35.139661 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1267/2530] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:35 2022] POST /identity/v3/users => generated 305 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:35.144480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c54ef3-062a-4f87-a745-de218965d3bb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.160310 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c54ef3-062a-4f87-a745-de218965d3bb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:35.161840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d10192-5031-42b7-8fc7-e3ab8fcbebf6 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.164110 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d1d10192-5031-42b7-8fc7-e3ab8fcbebf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:35.169181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf09e7d-6157-486f-827a-0510ff956d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:35.169892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf09e7d-6157-486f-827a-0510ff956d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.170139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf09e7d-6157-486f-827a-0510ff956d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.170413 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf09e7d-6157-486f-827a-0510ff956d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.171207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf09e7d-6157-486f-827a-0510ff956d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:35.172910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf09e7d-6157-486f-827a-0510ff956d24 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:35.183768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1264/2531] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:35.187685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf09e7d-6157-486f-827a-0510ff956d24 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.198248 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf09e7d-6157-486f-827a-0510ff956d24 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:35.203789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba890fd-f25d-4baa-beb0-c30804cb63d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:35.204857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba890fd-f25d-4baa-beb0-c30804cb63d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.205212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba890fd-f25d-4baa-beb0-c30804cb63d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.205425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba890fd-f25d-4baa-beb0-c30804cb63d7 admin admin] PATH_INFO: `/v3/projects/32eeff8b22a44703bf7b2dc973c9a0e4/users/e06cdbc238644d80b48503bced82e0c6/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.218350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:35.221838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba890fd-f25d-4baa-beb0-c30804cb63d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32eeff8b22a44703bf7b2dc973c9a0e4, user_id=e06cdbc238644d80b48503bced82e0c6, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:35.224619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba890fd-f25d-4baa-beb0-c30804cb63d7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:35.236671 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d10192-5031-42b7-8fc7-e3ab8fcbebf6 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:35.241083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646899a8-50c0-4b83-8ac6-786130dc031e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:35.241747 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646899a8-50c0-4b83-8ac6-786130dc031e service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.242452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646899a8-50c0-4b83-8ac6-786130dc031e service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.242761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646899a8-50c0-4b83-8ac6-786130dc031e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.245564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1265/2532] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:35 2022] PUT /identity/v3/projects/32eeff8b22a44703bf7b2dc973c9a0e4/users/e06cdbc238644d80b48503bced82e0c6/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:35.249281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba890fd-f25d-4baa-beb0-c30804cb63d7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.254712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646899a8-50c0-4b83-8ac6-786130dc031e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:35.256756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646899a8-50c0-4b83-8ac6-786130dc031e service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:35.271799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1268/2533] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 06:29:35.318592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba890fd-f25d-4baa-beb0-c30804cb63d7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:35.327720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e90d2-5b1d-4c19-8e33-48ab61f3c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:35.328538 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e90d2-5b1d-4c19-8e33-48ab61f3c07c admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.328882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e90d2-5b1d-4c19-8e33-48ab61f3c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.329222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e90d2-5b1d-4c19-8e33-48ab61f3c07c admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.329976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e90d2-5b1d-4c19-8e33-48ab61f3c07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:35.331678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e90d2-5b1d-4c19-8e33-48ab61f3c07c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:35.348894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1266/2534] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:35.349138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646899a8-50c0-4b83-8ac6-786130dc031e service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.361829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646899a8-50c0-4b83-8ac6-786130dc031e service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:35.368505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a07745-2c9d-44cb-812d-3b3a0244d9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:35.369453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a07745-2c9d-44cb-812d-3b3a0244d9da admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.369738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a07745-2c9d-44cb-812d-3b3a0244d9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.370012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a07745-2c9d-44cb-812d-3b3a0244d9da admin admin] PATH_INFO: `/v3/projects/32eeff8b22a44703bf7b2dc973c9a0e4/users/e06cdbc238644d80b48503bced82e0c6/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.373033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a07745-2c9d-44cb-812d-3b3a0244d9da admin admin] RBAC: Authorizing `identity:create_grant(project_id=32eeff8b22a44703bf7b2dc973c9a0e4, user_id=e06cdbc238644d80b48503bced82e0c6, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:35.375926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a07745-2c9d-44cb-812d-3b3a0244d9da admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:35.391276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1269/2535] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:35 2022] PUT /identity/v3/projects/32eeff8b22a44703bf7b2dc973c9a0e4/users/e06cdbc238644d80b48503bced82e0c6/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:35.402239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02610-4cb9-4e23-ad20-dda9b7617344 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.402804 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02610-4cb9-4e23-ad20-dda9b7617344 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.403101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02610-4cb9-4e23-ad20-dda9b7617344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.437624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4da02610-4cb9-4e23-ad20-dda9b7617344 None None] MFA Rules not processed for user `e06cdbc238644d80b48503bced82e0c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:35.438939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4da02610-4cb9-4e23-ad20-dda9b7617344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:35.447079 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a07745-2c9d-44cb-812d-3b3a0244d9da admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.553757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a07745-2c9d-44cb-812d-3b3a0244d9da admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:35.559469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1267/2536] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:35.560165 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:35.561192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.561548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.561884 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] PATH_INFO: `/v3/users/2771829ad13b4bb69a855eb04825d513` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.563445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2771829ad13b4bb69a855eb04825d513)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:35.566191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:35.600573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:35.612927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02610-4cb9-4e23-ad20-dda9b7617344 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.632697 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2771829ad13b4bb69a855eb04825d513'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:35.641038 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2771829ad13b4bb69a855eb04825d513'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:35.641753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2771829ad13b4bb69a855eb04825d513'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:35.652442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2771829ad13b4bb69a855eb04825d513'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:35.667137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2771829ad13b4bb69a855eb04825d513'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:35.673994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1270/2537] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:35 2022] DELETE /identity/v3/users/2771829ad13b4bb69a855eb04825d513 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:35.674870 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.680693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:35.693426 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02610-4cb9-4e23-ad20-dda9b7617344 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:35.718961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:35.742867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698d593-1ef9-4514-8368-1ef8f7b349b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:35.743711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698d593-1ef9-4514-8368-1ef8f7b349b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.744082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698d593-1ef9-4514-8368-1ef8f7b349b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.744394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698d593-1ef9-4514-8368-1ef8f7b349b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.746295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e698d593-1ef9-4514-8368-1ef8f7b349b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:35.746917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bd606-9891-47e6-aa78-82aeaee6b967 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:35.751820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:35.752682 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.752956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.753225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] PATH_INFO: `/v3/users/75bb30ebbcc14ce48b9fe0876dd8f3b4` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.754807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75bb30ebbcc14ce48b9fe0876dd8f3b4)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:35.756879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:35.808732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75bb30ebbcc14ce48b9fe0876dd8f3b4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:35.816906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75bb30ebbcc14ce48b9fe0876dd8f3b4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:35.817749 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75bb30ebbcc14ce48b9fe0876dd8f3b4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:35.820989 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698d593-1ef9-4514-8368-1ef8f7b349b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:35.823229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698d593-1ef9-4514-8368-1ef8f7b349b2 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:35.828189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75bb30ebbcc14ce48b9fe0876dd8f3b4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:35.836039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1268/2538] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:35.838475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9667c-77f5-416d-9961-7cf218ba8255 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.838644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9667c-77f5-416d-9961-7cf218ba8255 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.839038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9667c-77f5-416d-9961-7cf218ba8255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.840848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-eb49c0b4-daf1-47de-8a94-98ff1fd27a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75bb30ebbcc14ce48b9fe0876dd8f3b4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:35.851469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1271/2539] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:35 2022] DELETE /identity/v3/users/75bb30ebbcc14ce48b9fe0876dd8f3b4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:35.857567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add84722-e73d-4479-8efd-57d0dc58c2b7 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:35.858007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add84722-e73d-4479-8efd-57d0dc58c2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:35.858576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add84722-e73d-4479-8efd-57d0dc58c2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:35.871223 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-26a9667c-77f5-416d-9961-7cf218ba8255 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:35.872470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-26a9667c-77f5-416d-9961-7cf218ba8255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:35.883732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-add84722-e73d-4479-8efd-57d0dc58c2b7 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:35.885171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-add84722-e73d-4479-8efd-57d0dc58c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:35.916037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:35.952782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1269/2540] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:35.953635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9667c-77f5-416d-9961-7cf218ba8255 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.954880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-26a9667c-77f5-416d-9961-7cf218ba8255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:35.970225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1272/2541] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:35.971146 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add84722-e73d-4479-8efd-57d0dc58c2b7 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:35.974140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-add84722-e73d-4479-8efd-57d0dc58c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:36.016419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9667c-77f5-416d-9961-7cf218ba8255 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:36.020457 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946566d-354a-42d6-9c03-972b21b7903e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:36.021123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946566d-354a-42d6-9c03-972b21b7903e service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:36.021407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946566d-354a-42d6-9c03-972b21b7903e service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:36.021681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946566d-354a-42d6-9c03-972b21b7903e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:36.023533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f946566d-354a-42d6-9c03-972b21b7903e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:36.034033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add84722-e73d-4479-8efd-57d0dc58c2b7 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:36.037677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76359bf8-0c46-47ab-99ad-010bfb484e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:36.038336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76359bf8-0c46-47ab-99ad-010bfb484e59 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:36.038617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76359bf8-0c46-47ab-99ad-010bfb484e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:36.038893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76359bf8-0c46-47ab-99ad-010bfb484e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:36.048419 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76359bf8-0c46-47ab-99ad-010bfb484e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:36.049876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76359bf8-0c46-47ab-99ad-010bfb484e59 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:36.055971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:36.060139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1273/2542] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:36.060954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76359bf8-0c46-47ab-99ad-010bfb484e59 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:36.069410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76359bf8-0c46-47ab-99ad-010bfb484e59 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:36.073339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae70e17-b4ee-4320-bf24-58adb5ec712f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:36.073987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae70e17-b4ee-4320-bf24-58adb5ec712f service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:36.074271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae70e17-b4ee-4320-bf24-58adb5ec712f service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:36.074562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae70e17-b4ee-4320-bf24-58adb5ec712f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:36.084430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae70e17-b4ee-4320-bf24-58adb5ec712f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:36.085928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946566d-354a-42d6-9c03-972b21b7903e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:36.086073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae70e17-b4ee-4320-bf24-58adb5ec712f service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:36.087711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946566d-354a-42d6-9c03-972b21b7903e service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:36.097293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1274/2543] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:36.098689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1270/2544] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:36.477255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae70e17-b4ee-4320-bf24-58adb5ec712f service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:36.478894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4ae70e17-b4ee-4320-bf24-58adb5ec712f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:36.497262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae70e17-b4ee-4320-bf24-58adb5ec712f service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:36.528753 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:36.529979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:36.530348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:36.530629 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] PATH_INFO: `/v3/projects/e9bae52f7f3649f0839ec9176ccadecf` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:36.531951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9bae52f7f3649f0839ec9176ccadecf)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:36.533991 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:36.544653 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946566d-354a-42d6-9c03-972b21b7903e service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:36.546192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f946566d-354a-42d6-9c03-972b21b7903e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:36.567042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946566d-354a-42d6-9c03-972b21b7903e service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:36.573009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:36.574163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:36.574533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:36.574860 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] PATH_INFO: `/v3/projects/da4feb4c4ad04ad4a3fd031e4a78618e` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:36.576943 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da4feb4c4ad04ad4a3fd031e4a78618e)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:36.579336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:36.600417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9bae52f7f3649f0839ec9176ccadecf', 'request_id': 'req-09297571-dd7d-48cf-8253-a6f4af2322ba'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.607550 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9bae52f7f3649f0839ec9176ccadecf', 'request_id': 'req-09297571-dd7d-48cf-8253-a6f4af2322ba'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.613257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9bae52f7f3649f0839ec9176ccadecf', 'request_id': 'req-09297571-dd7d-48cf-8253-a6f4af2322ba'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.629453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] The token cache is being invalidate because project e9bae52f7f3649f0839ec9176ccadecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:36.629755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.630363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.632137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1275/2545] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:36 2022] DELETE /identity/v3/projects/e9bae52f7f3649f0839ec9176ccadecf => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:36.636285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:36.637760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:36.641016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da4feb4c4ad04ad4a3fd031e4a78618e', 'request_id': 'req-31073b1f-b121-4de5-8d9d-0f2377ecbad9'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.647537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da4feb4c4ad04ad4a3fd031e4a78618e', 'request_id': 'req-31073b1f-b121-4de5-8d9d-0f2377ecbad9'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.653378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da4feb4c4ad04ad4a3fd031e4a78618e', 'request_id': 'req-31073b1f-b121-4de5-8d9d-0f2377ecbad9'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.667210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] The token cache is being invalidate because project da4feb4c4ad04ad4a3fd031e4a78618e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:36.667824 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.668517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.670217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1271/2546] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:36 2022] DELETE /identity/v3/projects/da4feb4c4ad04ad4a3fd031e4a78618e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:36.674883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:36.676718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:36.683056 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:36.721235 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09297571-dd7d-48cf-8253-a6f4af2322ba admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:36.790228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31073b1f-b121-4de5-8d9d-0f2377ecbad9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:36.801237 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:36.802800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:36.803315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:36.803765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] PATH_INFO: `/v3/users/da99b1840cb64000910238f45e82ef65` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:36.804164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:36.805549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:36.805663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] RBAC: Authorizing `identity:delete_user(user_id=da99b1840cb64000910238f45e82ef65)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:36.806076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:36.806855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] PATH_INFO: `/v3/users/5e8d26c6fe034727ba1858b3064c7f5c` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:36.809224 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:36.809858 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e8d26c6fe034727ba1858b3064c7f5c)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:36.815157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:36.871126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da99b1840cb64000910238f45e82ef65'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.872006 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e8d26c6fe034727ba1858b3064c7f5c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.878193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da99b1840cb64000910238f45e82ef65'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.879122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e8d26c6fe034727ba1858b3064c7f5c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.879227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da99b1840cb64000910238f45e82ef65'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.881788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:36.891306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da99b1840cb64000910238f45e82ef65'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.894562 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e8d26c6fe034727ba1858b3064c7f5c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.895745 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e8d26c6fe034727ba1858b3064c7f5c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.896574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e8d26c6fe034727ba1858b3064c7f5c'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.905247 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-be2b9d28-92ee-483b-ae9a-8bee6695b64f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da99b1840cb64000910238f45e82ef65'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:36.911560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1276/2547] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:36 2022] DELETE /identity/v3/users/da99b1840cb64000910238f45e82ef65 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:36.914528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1272/2548] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:36 2022] DELETE /identity/v3/users/5e8d26c6fe034727ba1858b3064c7f5c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:36.916079 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:36.917577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:36.983592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc269-724c-465e-a5c2-13d209113102 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:36.988657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:36.989552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:36.989835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:36.990107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] PATH_INFO: `/v3/users/e387d72935434123a0b5dd6c9986711f` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:36.991373 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e387d72935434123a0b5dd6c9986711f)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:36.993319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:37.044590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e387d72935434123a0b5dd6c9986711f'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.052872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e387d72935434123a0b5dd6c9986711f'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.061773 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e387d72935434123a0b5dd6c9986711f'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.062693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e387d72935434123a0b5dd6c9986711f'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.063509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e387d72935434123a0b5dd6c9986711f'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.082107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1273/2549] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:36 2022] DELETE /identity/v3/users/e387d72935434123a0b5dd6c9986711f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:37.087366 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0191f-2df8-4325-ba64-94de44902118 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.087628 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0191f-2df8-4325-ba64-94de44902118 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.088223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0191f-2df8-4325-ba64-94de44902118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.105768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-1dd0191f-2df8-4325-ba64-94de44902118 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:37.107074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1dd0191f-2df8-4325-ba64-94de44902118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.180657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1277/2550] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:37.199730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:37.201261 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.285756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d74c81-8f79-4bcc-9680-42ab535bd74c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:37.290430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076e2b-3fe9-4ae1-940d-dc6fd962e1c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:37.291693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076e2b-3fe9-4ae1-940d-dc6fd962e1c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.292265 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076e2b-3fe9-4ae1-940d-dc6fd962e1c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.292680 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076e2b-3fe9-4ae1-940d-dc6fd962e1c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.294945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0191f-2df8-4325-ba64-94de44902118 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:37.297074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1dd0191f-2df8-4325-ba64-94de44902118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.304544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29076e2b-3fe9-4ae1-940d-dc6fd962e1c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:37.307026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29076e2b-3fe9-4ae1-940d-dc6fd962e1c9 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:37.309073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:37.313974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:37.319343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0191f-2df8-4325-ba64-94de44902118 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:37.326356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:37.327972 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.329908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.330411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] PATH_INFO: `/v3/projects/047b352008824dbba7ce6b4474d9566b` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.331585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1274/2551] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:37.332396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=047b352008824dbba7ce6b4474d9566b)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:37.336910 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:37.346471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076e2b-3fe9-4ae1-940d-dc6fd962e1c9 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:37.348157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-29076e2b-3fe9-4ae1-940d-dc6fd962e1c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.369642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076e2b-3fe9-4ae1-940d-dc6fd962e1c9 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:37.382911 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:37.384048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.384309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.384590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] PATH_INFO: `/v3/projects/eb531d64b84347c098dac87640bfee74` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.386845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb531d64b84347c098dac87640bfee74)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:37.389263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:37.403925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '047b352008824dbba7ce6b4474d9566b', 'request_id': 'req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.411829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '047b352008824dbba7ce6b4474d9566b', 'request_id': 'req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.419159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '047b352008824dbba7ce6b4474d9566b', 'request_id': 'req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.434965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] The token cache is being invalidate because project 047b352008824dbba7ce6b4474d9566b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:37.435436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.436167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b0b7e9aa-8302-421e-8a50-1c31eb9cdbfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.437859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1278/2552] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:37 2022] DELETE /identity/v3/projects/047b352008824dbba7ce6b4474d9566b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:37.442567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39881538-6f19-499e-9522-d4f62d966016 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.442926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39881538-6f19-499e-9522-d4f62d966016 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.443477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39881538-6f19-499e-9522-d4f62d966016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.452852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb531d64b84347c098dac87640bfee74', 'request_id': 'req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.459382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb531d64b84347c098dac87640bfee74', 'request_id': 'req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.465341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb531d64b84347c098dac87640bfee74', 'request_id': 'req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.469209 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-39881538-6f19-499e-9522-d4f62d966016 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:37.470925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-39881538-6f19-499e-9522-d4f62d966016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.482195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] The token cache is being invalidate because project eb531d64b84347c098dac87640bfee74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:37.482494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.483166 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-dd3d15c9-e27c-4e51-ae1c-57f2dd0d4109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.485046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1275/2553] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:37 2022] DELETE /identity/v3/projects/eb531d64b84347c098dac87640bfee74 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:37.489890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965edf3a-5229-49b3-97da-7271c678ab37 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.490170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965edf3a-5229-49b3-97da-7271c678ab37 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.490580 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965edf3a-5229-49b3-97da-7271c678ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.511037 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-965edf3a-5229-49b3-97da-7271c678ab37 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:37.512329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-965edf3a-5229-49b3-97da-7271c678ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.545296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1279/2554] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:37.553089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057145d-6d3f-479b-98c8-9b0e68420023 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.553382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057145d-6d3f-479b-98c8-9b0e68420023 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.553660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057145d-6d3f-479b-98c8-9b0e68420023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.575532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a057145d-6d3f-479b-98c8-9b0e68420023 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:37.576294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a057145d-6d3f-479b-98c8-9b0e68420023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.588355 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1280/2555] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:37.589508 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057145d-6d3f-479b-98c8-9b0e68420023 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:37.591679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a057145d-6d3f-479b-98c8-9b0e68420023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.637266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1276/2556] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:37.638451 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965edf3a-5229-49b3-97da-7271c678ab37 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:37.658020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965edf3a-5229-49b3-97da-7271c678ab37 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:37.659820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057145d-6d3f-479b-98c8-9b0e68420023 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:37.663922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f9f5c-2b1b-4220-af59-a387f5077649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:37.664590 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f9f5c-2b1b-4220-af59-a387f5077649 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.665170 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f9f5c-2b1b-4220-af59-a387f5077649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.665350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f9f5c-2b1b-4220-af59-a387f5077649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.665859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc3bec-836b-4d7a-b1f7-bf0e573f89a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:37.666542 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc3bec-836b-4d7a-b1f7-bf0e573f89a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.667005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc3bec-836b-4d7a-b1f7-bf0e573f89a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.667318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc3bec-836b-4d7a-b1f7-bf0e573f89a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.668515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cf1f9f5c-2b1b-4220-af59-a387f5077649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.668629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc3bec-836b-4d7a-b1f7-bf0e573f89a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:37.668831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc3bec-836b-4d7a-b1f7-bf0e573f89a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:37.671685 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc3bec-836b-4d7a-b1f7-bf0e573f89a9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:37.692101 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1277/2557] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:29:37.692208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f9f5c-2b1b-4220-af59-a387f5077649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:37.692328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1406aa2-3f9c-4183-bfd0-f0201c2a6280 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.692328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1406aa2-3f9c-4183-bfd0-f0201c2a6280 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.692328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1406aa2-3f9c-4183-bfd0-f0201c2a6280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.692592 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f9f5c-2b1b-4220-af59-a387f5077649 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:37.710605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1281/2558] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:37.711968 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f9f5c-2b1b-4220-af59-a387f5077649 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:37.713587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cf1f9f5c-2b1b-4220-af59-a387f5077649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.718411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a1406aa2-3f9c-4183-bfd0-f0201c2a6280 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:37.719794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a1406aa2-3f9c-4183-bfd0-f0201c2a6280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.727456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f9f5c-2b1b-4220-af59-a387f5077649 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:37.733292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:37.733630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1278/2559] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:37.735336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1406aa2-3f9c-4183-bfd0-f0201c2a6280 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:37.735537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.736162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.736802 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] PATH_INFO: `/v3/projects/d4767021b5ca4e78a01e55280fced204` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.738226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4767021b5ca4e78a01e55280fced204)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:37.740865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:37.749115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1406aa2-3f9c-4183-bfd0-f0201c2a6280 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:37.755434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea4fac-7cd4-492f-bc80-781d6037b6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:37.757929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4fac-7cd4-492f-bc80-781d6037b6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.758201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4fac-7cd4-492f-bc80-781d6037b6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.758466 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4fac-7cd4-492f-bc80-781d6037b6c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.759437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea4fac-7cd4-492f-bc80-781d6037b6c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:37.761914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea4fac-7cd4-492f-bc80-781d6037b6c5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:37.765767 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:37.802065 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1279/2560] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:37 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:37.802065 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea4fac-7cd4-492f-bc80-781d6037b6c5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:37.815148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4767021b5ca4e78a01e55280fced204', 'request_id': 'req-d3ab31fa-8080-48df-9148-5858e5ad22ce'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.823297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4767021b5ca4e78a01e55280fced204', 'request_id': 'req-d3ab31fa-8080-48df-9148-5858e5ad22ce'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.830399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4767021b5ca4e78a01e55280fced204', 'request_id': 'req-d3ab31fa-8080-48df-9148-5858e5ad22ce'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.849928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] The token cache is being invalidate because project d4767021b5ca4e78a01e55280fced204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:37.850262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.850940 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:37.854190 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1282/2561] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:37 2022] DELETE /identity/v3/projects/d4767021b5ca4e78a01e55280fced204 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:37.855622 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:37.857201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.866115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:37.896471 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea4fac-7cd4-492f-bc80-781d6037b6c5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:37.903383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4990c-88fd-4a02-99e2-527ce9d42f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:37.904182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4990c-88fd-4a02-99e2-527ce9d42f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.904463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4990c-88fd-4a02-99e2-527ce9d42f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.904735 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4990c-88fd-4a02-99e2-527ce9d42f0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.905541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4990c-88fd-4a02-99e2-527ce9d42f0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:37.905887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4990c-88fd-4a02-99e2-527ce9d42f0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:37.908404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4990c-88fd-4a02-99e2-527ce9d42f0b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:37.930348 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1280/2562] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:29:37.932049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4990c-88fd-4a02-99e2-527ce9d42f0b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:37.942061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6af4990c-88fd-4a02-99e2-527ce9d42f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:37.957852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4990c-88fd-4a02-99e2-527ce9d42f0b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:37.964057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:37.964851 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.965163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.965442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] PATH_INFO: `/v3/users/06dae7a75934405fbafedaa6d60655d8` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.966967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06dae7a75934405fbafedaa6d60655d8)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:37.969360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:37.980151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab31fa-8080-48df-9148-5858e5ad22ce admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:37.987470 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc854a-3e8c-4088-9cac-680980f04630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:37.988692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc854a-3e8c-4088-9cac-680980f04630 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:37.989037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc854a-3e8c-4088-9cac-680980f04630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:37.989314 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc854a-3e8c-4088-9cac-680980f04630 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:37.990414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc854a-3e8c-4088-9cac-680980f04630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:37.992362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc854a-3e8c-4088-9cac-680980f04630 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.026638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1283/2563] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:37 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:38.027797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc854a-3e8c-4088-9cac-680980f04630 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.029949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c7cc854a-3e8c-4088-9cac-680980f04630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.033193 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06dae7a75934405fbafedaa6d60655d8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.045103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06dae7a75934405fbafedaa6d60655d8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.045456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc854a-3e8c-4088-9cac-680980f04630 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.050326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9127d0-7235-49be-ad48-39c74e68f496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.050996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9127d0-7235-49be-ad48-39c74e68f496 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.051306 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9127d0-7235-49be-ad48-39c74e68f496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.051591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9127d0-7235-49be-ad48-39c74e68f496 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.052296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9127d0-7235-49be-ad48-39c74e68f496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.054382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9127d0-7235-49be-ad48-39c74e68f496 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.055164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06dae7a75934405fbafedaa6d60655d8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.056301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06dae7a75934405fbafedaa6d60655d8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.057150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06dae7a75934405fbafedaa6d60655d8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.083281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1281/2564] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:37 2022] DELETE /identity/v3/users/06dae7a75934405fbafedaa6d60655d8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:38.084534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.085150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1284/2565] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:38 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:38.088757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.092425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9127d0-7235-49be-ad48-39c74e68f496 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.101035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1f9127d0-7235-49be-ad48-39c74e68f496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.169509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317cb59-0bb3-4685-aaca-02030ebd4b78 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.212344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9127d0-7235-49be-ad48-39c74e68f496 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.219412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a0963-405b-487c-9c18-f1bbfd032cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.220585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a0963-405b-487c-9c18-f1bbfd032cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.221008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a0963-405b-487c-9c18-f1bbfd032cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.221414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a0963-405b-487c-9c18-f1bbfd032cf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.222124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a0963-405b-487c-9c18-f1bbfd032cf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.224219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a0963-405b-487c-9c18-f1bbfd032cf0 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.228213 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:38.241546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854ba58-8dae-4f2a-b296-038c87ef4575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.243483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854ba58-8dae-4f2a-b296-038c87ef4575 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.243780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854ba58-8dae-4f2a-b296-038c87ef4575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.244181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854ba58-8dae-4f2a-b296-038c87ef4575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.245082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854ba58-8dae-4f2a-b296-038c87ef4575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.249834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854ba58-8dae-4f2a-b296-038c87ef4575 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.261537 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1285/2566] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:38 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:38.263100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a0963-405b-487c-9c18-f1bbfd032cf0 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.269440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=8 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:38.280506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a0963-405b-487c-9c18-f1bbfd032cf0 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.284794 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1282/2567] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:38.286485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854ba58-8dae-4f2a-b296-038c87ef4575 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.299061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa74ef-ddc4-4ab8-b1bd-b6ab8566f785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.299843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa74ef-ddc4-4ab8-b1bd-b6ab8566f785 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.300128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa74ef-ddc4-4ab8-b1bd-b6ab8566f785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.300411 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa74ef-ddc4-4ab8-b1bd-b6ab8566f785 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.301150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa74ef-ddc4-4ab8-b1bd-b6ab8566f785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.302990 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa74ef-ddc4-4ab8-b1bd-b6ab8566f785 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.306765 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854ba58-8dae-4f2a-b296-038c87ef4575 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.314442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b818-6487-4f72-a9bc-f8ce3c9f24a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.315749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b818-6487-4f72-a9bc-f8ce3c9f24a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.316265 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b818-6487-4f72-a9bc-f8ce3c9f24a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.317036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b818-6487-4f72-a9bc-f8ce3c9f24a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.317866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1286/2568] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:38.318978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa74ef-ddc4-4ab8-b1bd-b6ab8566f785 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.319293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146b818-6487-4f72-a9bc-f8ce3c9f24a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.321294 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146b818-6487-4f72-a9bc-f8ce3c9f24a2 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.333692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa74ef-ddc4-4ab8-b1bd-b6ab8566f785 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.336548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1283/2569] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:38.343116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b818-6487-4f72-a9bc-f8ce3c9f24a2 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.346721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb25a-3803-4326-a3fd-22f5f64ed91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.347637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb25a-3803-4326-a3fd-22f5f64ed91d admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.348615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb25a-3803-4326-a3fd-22f5f64ed91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.349072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb25a-3803-4326-a3fd-22f5f64ed91d admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.349983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcb25a-3803-4326-a3fd-22f5f64ed91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.353837 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcb25a-3803-4326-a3fd-22f5f64ed91d admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.361430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b818-6487-4f72-a9bc-f8ce3c9f24a2 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.366565 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fdbac-8859-47a8-a19f-b7ac2c09c339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.367514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fdbac-8859-47a8-a19f-b7ac2c09c339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.367927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fdbac-8859-47a8-a19f-b7ac2c09c339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.368266 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fdbac-8859-47a8-a19f-b7ac2c09c339 admin admin] PATH_INFO: `/v3/projects/3d7bdc049109419fb0b7188730aeef9f/users/d320afdb49a64af1b5dbdfdb52e37d90/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.369621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1287/2570] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:38.374139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb25a-3803-4326-a3fd-22f5f64ed91d admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.389091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb25a-3803-4326-a3fd-22f5f64ed91d admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.398395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fdbac-8859-47a8-a19f-b7ac2c09c339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7bdc049109419fb0b7188730aeef9f, user_id=d320afdb49a64af1b5dbdfdb52e37d90, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.398992 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff93e7-bd58-4636-b776-bd297cb127bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.400141 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff93e7-bd58-4636-b776-bd297cb127bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.400248 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff93e7-bd58-4636-b776-bd297cb127bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.400477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff93e7-bd58-4636-b776-bd297cb127bf admin admin] PATH_INFO: `/v3/projects/0187d6ec193c4599b0141054122b4550/users/f0176f4e60c24dd5a864544aef26972d/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.401201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fdbac-8859-47a8-a19f-b7ac2c09c339 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.417669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1284/2571] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:38 2022] PUT /identity/v3/projects/3d7bdc049109419fb0b7188730aeef9f/users/d320afdb49a64af1b5dbdfdb52e37d90/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:38.419013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fdbac-8859-47a8-a19f-b7ac2c09c339 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.421001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b30fdbac-8859-47a8-a19f-b7ac2c09c339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.428705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff93e7-bd58-4636-b776-bd297cb127bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0187d6ec193c4599b0141054122b4550, user_id=f0176f4e60c24dd5a864544aef26972d, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.431652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff93e7-bd58-4636-b776-bd297cb127bf admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.449635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1288/2572] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:38 2022] PUT /identity/v3/projects/0187d6ec193c4599b0141054122b4550/users/f0176f4e60c24dd5a864544aef26972d/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:38.459645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f9c3d-4720-4647-b093-1ab8a9f94e8d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.459936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f9c3d-4720-4647-b093-1ab8a9f94e8d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.460298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f9c3d-4720-4647-b093-1ab8a9f94e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.487054 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c05f9c3d-4720-4647-b093-1ab8a9f94e8d None None] MFA Rules not processed for user `d320afdb49a64af1b5dbdfdb52e37d90`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:38.488336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c05f9c3d-4720-4647-b093-1ab8a9f94e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.504454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fdbac-8859-47a8-a19f-b7ac2c09c339 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.552944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1289/2573] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:38.553883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f9c3d-4720-4647-b093-1ab8a9f94e8d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.555423 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c05f9c3d-4720-4647-b093-1ab8a9f94e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.559541 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.560184 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.560601 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.560884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] PATH_INFO: `/v3/users/e78b454f9d0b4e99ae0425cd596f02c0` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.562169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e78b454f9d0b4e99ae0425cd596f02c0)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.564579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.569638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f9c3d-4720-4647-b093-1ab8a9f94e8d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.574331 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.575226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.575534 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.575814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] PATH_INFO: `/v3/projects/7d6ea27e5f7448c1a159a72de27cd47d` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.576978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d6ea27e5f7448c1a159a72de27cd47d)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.578888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.621412 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e78b454f9d0b4e99ae0425cd596f02c0'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.629800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e78b454f9d0b4e99ae0425cd596f02c0'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.632134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d6ea27e5f7448c1a159a72de27cd47d', 'request_id': 'req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.638522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d6ea27e5f7448c1a159a72de27cd47d', 'request_id': 'req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.638678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e78b454f9d0b4e99ae0425cd596f02c0'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.639445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e78b454f9d0b4e99ae0425cd596f02c0'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.640201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e78b454f9d0b4e99ae0425cd596f02c0'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.641610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:38.644982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d6ea27e5f7448c1a159a72de27cd47d', 'request_id': 'req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.661737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] The token cache is being invalidate because project 7d6ea27e5f7448c1a159a72de27cd47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:38.662028 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.662884 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-df7b8e0d-bd1a-44fe-95bb-840ab92ee320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.667426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1290/2574] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:38 2022] DELETE /identity/v3/projects/7d6ea27e5f7448c1a159a72de27cd47d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:38.667590 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1285/2575] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:38 2022] DELETE /identity/v3/users/e78b454f9d0b4e99ae0425cd596f02c0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:38.667691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7231bc-c244-44b9-a6e9-1d89f17fa647 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.667778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.667861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7231bc-c244-44b9-a6e9-1d89f17fa647 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.667861 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7231bc-c244-44b9-a6e9-1d89f17fa647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.669186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.699144 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-cd7231bc-c244-44b9-a6e9-1d89f17fa647 None None] MFA Rules not processed for user `f0176f4e60c24dd5a864544aef26972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:38.700622 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cd7231bc-c244-44b9-a6e9-1d89f17fa647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.721656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:38.741405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7518b1-3caf-481e-a5cd-d824564d625c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.746865 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4e107-b057-43bf-a09e-f343370b4d86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.747587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4e107-b057-43bf-a09e-f343370b4d86 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.747876 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4e107-b057-43bf-a09e-f343370b4d86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.748156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4e107-b057-43bf-a09e-f343370b4d86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.750215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-17b4e107-b057-43bf-a09e-f343370b4d86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.777137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1291/2576] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:38.778360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7231bc-c244-44b9-a6e9-1d89f17fa647 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.781981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cd7231bc-c244-44b9-a6e9-1d89f17fa647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.824399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4e107-b057-43bf-a09e-f343370b4d86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.825863 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4e107-b057-43bf-a09e-f343370b4d86 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.835913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1286/2577] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:38.836753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4e107-b057-43bf-a09e-f343370b4d86 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.838174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-17b4e107-b057-43bf-a09e-f343370b4d86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.848942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4e107-b057-43bf-a09e-f343370b4d86 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.853286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13b750-197a-4bb2-b820-c3b33f1d9f7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.853996 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13b750-197a-4bb2-b820-c3b33f1d9f7e service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.854282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13b750-197a-4bb2-b820-c3b33f1d9f7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.854755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13b750-197a-4bb2-b820-c3b33f1d9f7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.859938 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7231bc-c244-44b9-a6e9-1d89f17fa647 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.864585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13b750-197a-4bb2-b820-c3b33f1d9f7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.865319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.866469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.866843 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.867230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] PATH_INFO: `/v3/users/741a75546d5a478ba4ace7f5d2da34d9` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.869203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13b750-197a-4bb2-b820-c3b33f1d9f7e service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.869319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=741a75546d5a478ba4ace7f5d2da34d9)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.871461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.880107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1287/2578] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:38.880936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13b750-197a-4bb2-b820-c3b33f1d9f7e service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:38.882318 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4d13b750-197a-4bb2-b820-c3b33f1d9f7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:38.902107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13b750-197a-4bb2-b820-c3b33f1d9f7e service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:38.917503 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:38.918081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:38.918362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:38.918660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] PATH_INFO: `/v3/projects/b3ee0596be3b4648bd859293e02b1944` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:38.919915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ee0596be3b4648bd859293e02b1944)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:38.921867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:38.982294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '741a75546d5a478ba4ace7f5d2da34d9'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:38.997810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ee0596be3b4648bd859293e02b1944', 'request_id': 'req-20e58fac-c6a0-45b4-ba54-500b943adf3e'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.002601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '741a75546d5a478ba4ace7f5d2da34d9'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.003782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '741a75546d5a478ba4ace7f5d2da34d9'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.006461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ee0596be3b4648bd859293e02b1944', 'request_id': 'req-20e58fac-c6a0-45b4-ba54-500b943adf3e'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.012769 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ee0596be3b4648bd859293e02b1944', 'request_id': 'req-20e58fac-c6a0-45b4-ba54-500b943adf3e'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.014456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '741a75546d5a478ba4ace7f5d2da34d9'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.030665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] The token cache is being invalidate because project b3ee0596be3b4648bd859293e02b1944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:39.030965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.031653 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.033625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1288/2579] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:38 2022] DELETE /identity/v3/projects/b3ee0596be3b4648bd859293e02b1944 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:39.035090 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.035806 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-65879ffc-43de-4345-a87c-d0d05f24c957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '741a75546d5a478ba4ace7f5d2da34d9'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.036699 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.044114 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:39.049824 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1292/2580] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:38 2022] DELETE /identity/v3/users/741a75546d5a478ba4ace7f5d2da34d9 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:39.051970 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b7301-e51a-4145-9cf8-b756573b5a49 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.052524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b7301-e51a-4145-9cf8-b756573b5a49 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.052810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b7301-e51a-4145-9cf8-b756573b5a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.084007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-425b7301-e51a-4145-9cf8-b756573b5a49 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:39.085474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-425b7301-e51a-4145-9cf8-b756573b5a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.137855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e58fac-c6a0-45b4-ba54-500b943adf3e admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.142595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:39.204664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1293/2581] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:39.212840 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b1700-1730-461a-bcec-c7ca622e6e32 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.213258 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b1700-1730-461a-bcec-c7ca622e6e32 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.213621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b1700-1730-461a-bcec-c7ca622e6e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.254013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-e62b1700-1730-461a-bcec-c7ca622e6e32 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:39.255794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e62b1700-1730-461a-bcec-c7ca622e6e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.265927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1294/2582] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:39.266855 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.267833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.269759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.270047 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] PATH_INFO: `/v3/users/f0176f4e60c24dd5a864544aef26972d` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.270505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b1700-1730-461a-bcec-c7ca622e6e32 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.275012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0176f4e60c24dd5a864544aef26972d)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.278230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.294273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b1700-1730-461a-bcec-c7ca622e6e32 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.301068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a1f7c-664e-4ac6-9ef9-2360f8cc476f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.301799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a1f7c-664e-4ac6-9ef9-2360f8cc476f admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.302307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a1f7c-664e-4ac6-9ef9-2360f8cc476f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.302632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a1f7c-664e-4ac6-9ef9-2360f8cc476f admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.304346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a1f7c-664e-4ac6-9ef9-2360f8cc476f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:39.304658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a1f7c-664e-4ac6-9ef9-2360f8cc476f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.311838 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a1f7c-664e-4ac6-9ef9-2360f8cc476f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.333683 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1295/2583] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 06:29:39.338087 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a1f7c-664e-4ac6-9ef9-2360f8cc476f admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.354114 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0176f4e60c24dd5a864544aef26972d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.365024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0176f4e60c24dd5a864544aef26972d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.366413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a1f7c-664e-4ac6-9ef9-2360f8cc476f admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.378705 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0176f4e60c24dd5a864544aef26972d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.380360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0176f4e60c24dd5a864544aef26972d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.383517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-a2e6d0d2-ccaf-4a68-aaf0-8a224ea64099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0176f4e60c24dd5a864544aef26972d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.386178 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb601b01-f4a9-44ed-a4ad-c1bb6fbac301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.387188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb601b01-f4a9-44ed-a4ad-c1bb6fbac301 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.387351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb601b01-f4a9-44ed-a4ad-c1bb6fbac301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.387734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb601b01-f4a9-44ed-a4ad-c1bb6fbac301 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.388533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb601b01-f4a9-44ed-a4ad-c1bb6fbac301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.391692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb601b01-f4a9-44ed-a4ad-c1bb6fbac301 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.411034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1289/2584] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:39 2022] DELETE /identity/v3/users/f0176f4e60c24dd5a864544aef26972d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:39.416499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98148a62-af66-4c64-83ea-b6cf047d4ab2 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.417138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98148a62-af66-4c64-83ea-b6cf047d4ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.417592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98148a62-af66-4c64-83ea-b6cf047d4ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.440487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1296/2585] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:39 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:39.440712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-98148a62-af66-4c64-83ea-b6cf047d4ab2 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:39.440712 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-98148a62-af66-4c64-83ea-b6cf047d4ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.441569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb601b01-f4a9-44ed-a4ad-c1bb6fbac301 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.443952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-eb601b01-f4a9-44ed-a4ad-c1bb6fbac301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.471517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:39.515295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1290/2586] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:39.515530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98148a62-af66-4c64-83ea-b6cf047d4ab2 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.516931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-98148a62-af66-4c64-83ea-b6cf047d4ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.527131 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb601b01-f4a9-44ed-a4ad-c1bb6fbac301 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.530290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98148a62-af66-4c64-83ea-b6cf047d4ab2 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.532213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0601a9c-18a2-41aa-b2a4-d241504d323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.532963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0601a9c-18a2-41aa-b2a4-d241504d323e admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.533308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0601a9c-18a2-41aa-b2a4-d241504d323e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.533669 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0601a9c-18a2-41aa-b2a4-d241504d323e admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.534428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0601a9c-18a2-41aa-b2a4-d241504d323e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.535030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.535981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.536258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.536559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] PATH_INFO: `/v3/projects/af9bf37173154fe186166a07464db4c8` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.536723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0601a9c-18a2-41aa-b2a4-d241504d323e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.537799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af9bf37173154fe186166a07464db4c8)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.539852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.558881 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1297/2587] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:39 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:39.560059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0601a9c-18a2-41aa-b2a4-d241504d323e admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.562050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a0601a9c-18a2-41aa-b2a4-d241504d323e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.594508 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9bf37173154fe186166a07464db4c8', 'request_id': 'req-91b4f00d-db41-4334-a9ce-f14d0c363766'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.601619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9bf37173154fe186166a07464db4c8', 'request_id': 'req-91b4f00d-db41-4334-a9ce-f14d0c363766'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.607204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9bf37173154fe186166a07464db4c8', 'request_id': 'req-91b4f00d-db41-4334-a9ce-f14d0c363766'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.621095 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0601a9c-18a2-41aa-b2a4-d241504d323e admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.622664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] The token cache is being invalidate because project af9bf37173154fe186166a07464db4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:39.622959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.623659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:39.625258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1291/2588] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:39 2022] DELETE /identity/v3/projects/af9bf37173154fe186166a07464db4c8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:39.626024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.630600 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.667288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:39.679929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fcd67-0405-4e62-b7a1-a6faa939ddc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.680594 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fcd67-0405-4e62-b7a1-a6faa939ddc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.680874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fcd67-0405-4e62-b7a1-a6faa939ddc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.681148 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fcd67-0405-4e62-b7a1-a6faa939ddc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.683745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7b3fcd67-0405-4e62-b7a1-a6faa939ddc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.705366 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4f00d-db41-4334-a9ce-f14d0c363766 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.710108 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d89e3a-773c-4b00-a7c4-d5484c0d6b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.710852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89e3a-773c-4b00-a7c4-d5484c0d6b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.711064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89e3a-773c-4b00-a7c4-d5484c0d6b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.711393 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89e3a-773c-4b00-a7c4-d5484c0d6b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.712092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d89e3a-773c-4b00-a7c4-d5484c0d6b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.713729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d89e3a-773c-4b00-a7c4-d5484c0d6b2a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.727903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1292/2589] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:39.727903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc210975-03fa-4ef1-aa62-6cc689767e68 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.727903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc210975-03fa-4ef1-aa62-6cc689767e68 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.727903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc210975-03fa-4ef1-aa62-6cc689767e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.746853 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-fc210975-03fa-4ef1-aa62-6cc689767e68 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:39.748080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fc210975-03fa-4ef1-aa62-6cc689767e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.753654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fcd67-0405-4e62-b7a1-a6faa939ddc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.755108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fcd67-0405-4e62-b7a1-a6faa939ddc5 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.756167 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1293/2590] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:39.756939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc210975-03fa-4ef1-aa62-6cc689767e68 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.766103 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc210975-03fa-4ef1-aa62-6cc689767e68 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.769348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1298/2591] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:39.770690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501e65b-2954-4730-a9e5-4cf710994f9d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.770806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e2aa3-f3c2-490c-af12-6bb401d8d868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.770939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501e65b-2954-4730-a9e5-4cf710994f9d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.771220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501e65b-2954-4730-a9e5-4cf710994f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.771484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e2aa3-f3c2-490c-af12-6bb401d8d868 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.771767 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e2aa3-f3c2-490c-af12-6bb401d8d868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.772040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e2aa3-f3c2-490c-af12-6bb401d8d868 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.772785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e2aa3-f3c2-490c-af12-6bb401d8d868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.774561 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e2aa3-f3c2-490c-af12-6bb401d8d868 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.786718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1294/2592] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:39.790569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-9501e65b-2954-4730-a9e5-4cf710994f9d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:39.791330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e2aa3-f3c2-490c-af12-6bb401d8d868 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.792149 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9501e65b-2954-4730-a9e5-4cf710994f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.801324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1299/2593] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:39.802942 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e2aa3-f3c2-490c-af12-6bb401d8d868 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.805472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501e65b-2954-4730-a9e5-4cf710994f9d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.810837 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e31cd-106c-41dc-865e-7610d6fcc573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.812873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e31cd-106c-41dc-865e-7610d6fcc573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.813368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e31cd-106c-41dc-865e-7610d6fcc573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.813728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e31cd-106c-41dc-865e-7610d6fcc573 admin admin] PATH_INFO: `/v3/projects/574dca6a9bf744698378feacb0c10c86/users/444d988d8982463bb37a9a51950705d5/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.815835 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501e65b-2954-4730-a9e5-4cf710994f9d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.820518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e943efc-05ce-4b60-9996-59f9c7ab4ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.821191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e943efc-05ce-4b60-9996-59f9c7ab4ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.821473 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e943efc-05ce-4b60-9996-59f9c7ab4ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.821743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e943efc-05ce-4b60-9996-59f9c7ab4ba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.822471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e943efc-05ce-4b60-9996-59f9c7ab4ba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:39.822779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e943efc-05ce-4b60-9996-59f9c7ab4ba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.824625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e943efc-05ce-4b60-9996-59f9c7ab4ba4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.833670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e31cd-106c-41dc-865e-7610d6fcc573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=574dca6a9bf744698378feacb0c10c86, user_id=444d988d8982463bb37a9a51950705d5, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.837892 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1300/2594] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:29:39.838170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e31cd-106c-41dc-865e-7610d6fcc573 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.842515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e943efc-05ce-4b60-9996-59f9c7ab4ba4 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.853337 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e943efc-05ce-4b60-9996-59f9c7ab4ba4 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:39.862446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1295/2595] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:39 2022] PUT /identity/v3/projects/574dca6a9bf744698378feacb0c10c86/users/444d988d8982463bb37a9a51950705d5/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:39.865244 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85f094-9c8b-4de5-970e-c72778b6a08d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.865637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85f094-9c8b-4de5-970e-c72778b6a08d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.866015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85f094-9c8b-4de5-970e-c72778b6a08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.884727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:39.896203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ef85f094-9c8b-4de5-970e-c72778b6a08d None None] MFA Rules not processed for user `444d988d8982463bb37a9a51950705d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:39.898482 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ef85f094-9c8b-4de5-970e-c72778b6a08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.929490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3530a-05d8-4e38-a431-bf7e7e477032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:39.931635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3530a-05d8-4e38-a431-bf7e7e477032 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:39.932441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3530a-05d8-4e38-a431-bf7e7e477032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:39.932769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3530a-05d8-4e38-a431-bf7e7e477032 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:39.937187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3530a-05d8-4e38-a431-bf7e7e477032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:39.940406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3530a-05d8-4e38-a431-bf7e7e477032 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:39.971186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1296/2596] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:39.974333 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1301/2597] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:39 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:39.978535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85f094-9c8b-4de5-970e-c72778b6a08d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.982059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3530a-05d8-4e38-a431-bf7e7e477032 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:39.984162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e5f3530a-05d8-4e38-a431-bf7e7e477032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:39.990442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85f094-9c8b-4de5-970e-c72778b6a08d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.056504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3530a-05d8-4e38-a431-bf7e7e477032 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.060589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414149-e8e0-4f1b-8da1-db3a79c2d6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.061308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414149-e8e0-4f1b-8da1-db3a79c2d6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.061616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414149-e8e0-4f1b-8da1-db3a79c2d6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.061912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414149-e8e0-4f1b-8da1-db3a79c2d6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.073345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:40.073757 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412be96d-9cab-409c-bdee-430437a53c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.074421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412be96d-9cab-409c-bdee-430437a53c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.074768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412be96d-9cab-409c-bdee-430437a53c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.075509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412be96d-9cab-409c-bdee-430437a53c83 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.076317 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412be96d-9cab-409c-bdee-430437a53c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.078361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412be96d-9cab-409c-bdee-430437a53c83 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.102328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1297/2598] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:39 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:40.103121 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412be96d-9cab-409c-bdee-430437a53c83 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.105031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-412be96d-9cab-409c-bdee-430437a53c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:40.117596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412be96d-9cab-409c-bdee-430437a53c83 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.122536 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.123446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.123730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.124001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] PATH_INFO: `/v3/projects/0187d6ec193c4599b0141054122b4550` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.125162 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0187d6ec193c4599b0141054122b4550)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.127208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.133626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a414149-e8e0-4f1b-8da1-db3a79c2d6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.135106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a414149-e8e0-4f1b-8da1-db3a79c2d6f4 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.144025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1302/2599] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 06:29:40.144825 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414149-e8e0-4f1b-8da1-db3a79c2d6f4 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.154945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414149-e8e0-4f1b-8da1-db3a79c2d6f4 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.160134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ae3f9-d132-4eab-91b5-83780dc8a145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.160867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ae3f9-d132-4eab-91b5-83780dc8a145 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.161251 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ae3f9-d132-4eab-91b5-83780dc8a145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.161576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ae3f9-d132-4eab-91b5-83780dc8a145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.162431 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ae3f9-d132-4eab-91b5-83780dc8a145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.164240 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ae3f9-d132-4eab-91b5-83780dc8a145 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.175849 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1303/2600] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:40.179873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ae3f9-d132-4eab-91b5-83780dc8a145 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.190325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ae3f9-d132-4eab-91b5-83780dc8a145 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.192710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0187d6ec193c4599b0141054122b4550', 'request_id': 'req-e5715ed1-dbab-4399-b992-9f9cf2b8c111'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:40.200034 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0187d6ec193c4599b0141054122b4550', 'request_id': 'req-e5715ed1-dbab-4399-b992-9f9cf2b8c111'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:40.206299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0187d6ec193c4599b0141054122b4550', 'request_id': 'req-e5715ed1-dbab-4399-b992-9f9cf2b8c111'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:40.224822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] The token cache is being invalidate because project 0187d6ec193c4599b0141054122b4550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:40.225372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:40.226071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e5715ed1-dbab-4399-b992-9f9cf2b8c111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:40.228236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1298/2601] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:40 2022] DELETE /identity/v3/projects/0187d6ec193c4599b0141054122b4550 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:40.232831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a13fa-21dd-4d9a-b99c-f426fee593ee None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.233126 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a13fa-21dd-4d9a-b99c-f426fee593ee None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.233405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a13fa-21dd-4d9a-b99c-f426fee593ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.252869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4d0a13fa-21dd-4d9a-b99c-f426fee593ee None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:40.254580 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4d0a13fa-21dd-4d9a-b99c-f426fee593ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:40.255691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d67c08-1580-4ecc-adf1-4b316fc364ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.256351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d67c08-1580-4ecc-adf1-4b316fc364ef admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.256635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d67c08-1580-4ecc-adf1-4b316fc364ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.257425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d67c08-1580-4ecc-adf1-4b316fc364ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.257737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d67c08-1580-4ecc-adf1-4b316fc364ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.259499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d67c08-1580-4ecc-adf1-4b316fc364ef admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.263570 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1299/2602] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:40.272356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8339d9-0587-411d-909f-aaa58ba844c3 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.272864 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8339d9-0587-411d-909f-aaa58ba844c3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.273153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8339d9-0587-411d-909f-aaa58ba844c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.277382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1304/2603] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:40.281632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d67c08-1580-4ecc-adf1-4b316fc364ef admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.284560 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-55d67c08-1580-4ecc-adf1-4b316fc364ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:40.285360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:40.299627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ae8339d9-0587-411d-909f-aaa58ba844c3 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:40.301816 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ae8339d9-0587-411d-909f-aaa58ba844c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:40.305828 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d67c08-1580-4ecc-adf1-4b316fc364ef admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.312295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8d602-1161-458a-9e78-64aed8d56f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.313564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1300/2604] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:40.317410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8d602-1161-458a-9e78-64aed8d56f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.317734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8d602-1161-458a-9e78-64aed8d56f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.318024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8d602-1161-458a-9e78-64aed8d56f42 admin admin] PATH_INFO: `/v3/projects/654d19afeb8a468199281ad46f9217cb/users/efaa9f10ab4d41459c22bcb8cf960368/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.319220 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8339d9-0587-411d-909f-aaa58ba844c3 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.333177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8339d9-0587-411d-909f-aaa58ba844c3 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.335587 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8d602-1161-458a-9e78-64aed8d56f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654d19afeb8a468199281ad46f9217cb, user_id=efaa9f10ab4d41459c22bcb8cf960368, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.338397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8d602-1161-458a-9e78-64aed8d56f42 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.345790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5497c-c804-416e-9c46-86bdb0cf0635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.346770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5497c-c804-416e-9c46-86bdb0cf0635 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.347164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5497c-c804-416e-9c46-86bdb0cf0635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.347546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5497c-c804-416e-9c46-86bdb0cf0635 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.348555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5497c-c804-416e-9c46-86bdb0cf0635 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:40.348970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5497c-c804-416e-9c46-86bdb0cf0635 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.351024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5497c-c804-416e-9c46-86bdb0cf0635 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.362281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1305/2605] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:40 2022] PUT /identity/v3/projects/654d19afeb8a468199281ad46f9217cb/users/efaa9f10ab4d41459c22bcb8cf960368/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:40.369833 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1301/2606] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:29:40.373784 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8d602-1161-458a-9e78-64aed8d56f42 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.375773 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10baeeee-c3a0-4cc7-9c21-7547fc3dff52 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.376057 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10baeeee-c3a0-4cc7-9c21-7547fc3dff52 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.376330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10baeeee-c3a0-4cc7-9c21-7547fc3dff52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.414236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-10baeeee-c3a0-4cc7-9c21-7547fc3dff52 None None] MFA Rules not processed for user `efaa9f10ab4d41459c22bcb8cf960368`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:40.415756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-10baeeee-c3a0-4cc7-9c21-7547fc3dff52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:40.466342 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8d602-1161-458a-9e78-64aed8d56f42 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.473000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a1b81-f5f5-40ab-96fd-77c54b0b40c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.475312 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a1b81-f5f5-40ab-96fd-77c54b0b40c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.475535 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a1b81-f5f5-40ab-96fd-77c54b0b40c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.476023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a1b81-f5f5-40ab-96fd-77c54b0b40c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.476880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a1b81-f5f5-40ab-96fd-77c54b0b40c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.479012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a1b81-f5f5-40ab-96fd-77c54b0b40c8 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.522145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:40.530771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1302/2607] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:40.535967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10baeeee-c3a0-4cc7-9c21-7547fc3dff52 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.543297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1306/2608] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:40 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:40.546949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a1b81-f5f5-40ab-96fd-77c54b0b40c8 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.562042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10baeeee-c3a0-4cc7-9c21-7547fc3dff52 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.652195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a1b81-f5f5-40ab-96fd-77c54b0b40c8 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.659935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc65f24-2a8d-4f08-ada0-723d28ccf3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.661556 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc65f24-2a8d-4f08-ada0-723d28ccf3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.661993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc65f24-2a8d-4f08-ada0-723d28ccf3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.662463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc65f24-2a8d-4f08-ada0-723d28ccf3b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.663402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc65f24-2a8d-4f08-ada0-723d28ccf3b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.666374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc65f24-2a8d-4f08-ada0-723d28ccf3b9 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.697147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1307/2609] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:40 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:40.703851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc65f24-2a8d-4f08-ada0-723d28ccf3b9 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.704123 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e7b5b-9b6c-424b-adc0-ee3e4963fb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.706344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e7b5b-9b6c-424b-adc0-ee3e4963fb62 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.707116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e7b5b-9b6c-424b-adc0-ee3e4963fb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.707116 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e7b5b-9b6c-424b-adc0-ee3e4963fb62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.707811 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e7b5b-9b6c-424b-adc0-ee3e4963fb62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.710105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e7b5b-9b6c-424b-adc0-ee3e4963fb62 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.717363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc65f24-2a8d-4f08-ada0-723d28ccf3b9 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.730582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f0230-b7a9-4a60-a27c-1c3d6ef8ac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.731827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0230-b7a9-4a60-a27c-1c3d6ef8ac4d admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.732399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0230-b7a9-4a60-a27c-1c3d6ef8ac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.733126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0230-b7a9-4a60-a27c-1c3d6ef8ac4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.734987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f0230-b7a9-4a60-a27c-1c3d6ef8ac4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.737324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f0230-b7a9-4a60-a27c-1c3d6ef8ac4d admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.751309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1303/2610] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:40 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:40.752932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e7b5b-9b6c-424b-adc0-ee3e4963fb62 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.757639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1308/2611] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:40.761880 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f0230-b7a9-4a60-a27c-1c3d6ef8ac4d admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.775692 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f0230-b7a9-4a60-a27c-1c3d6ef8ac4d admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.833331 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e7b5b-9b6c-424b-adc0-ee3e4963fb62 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.837970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280124f-62b1-4d8d-8d1b-6ea04dc6105a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.838673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280124f-62b1-4d8d-8d1b-6ea04dc6105a admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.838986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280124f-62b1-4d8d-8d1b-6ea04dc6105a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.839302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280124f-62b1-4d8d-8d1b-6ea04dc6105a admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.839976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280124f-62b1-4d8d-8d1b-6ea04dc6105a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.841907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280124f-62b1-4d8d-8d1b-6ea04dc6105a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.843052 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd4f87-3878-4f86-a722-885ae652f92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.843711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd4f87-3878-4f86-a722-885ae652f92e admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.843993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd4f87-3878-4f86-a722-885ae652f92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.844269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd4f87-3878-4f86-a722-885ae652f92e admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.845009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd4f87-3878-4f86-a722-885ae652f92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.846610 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd4f87-3878-4f86-a722-885ae652f92e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.849017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:40.860664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1309/2612] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:40.860664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd4f87-3878-4f86-a722-885ae652f92e admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.860664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-3afd4f87-3878-4f86-a722-885ae652f92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:40.867291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1304/2613] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:40 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:40.868162 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280124f-62b1-4d8d-8d1b-6ea04dc6105a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.872817 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd4f87-3878-4f86-a722-885ae652f92e admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.877582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.878205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280124f-62b1-4d8d-8d1b-6ea04dc6105a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:40.878807 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.879087 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.879396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] PATH_INFO: `/v3/users/1bea5d0fe6d34482b85ac668fe33fb73` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.880574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bea5d0fe6d34482b85ac668fe33fb73)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.882511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.883223 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b82f7-ac69-42d3-8f6b-26bab1c5ada4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:40.884153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b82f7-ac69-42d3-8f6b-26bab1c5ada4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.884434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b82f7-ac69-42d3-8f6b-26bab1c5ada4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.884991 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b82f7-ac69-42d3-8f6b-26bab1c5ada4 admin admin] PATH_INFO: `/v3/projects/5eaf4b3186ab472da0cd67bf31fd5b03/users/c2fa14b07aa842f3b00cf1809ce676de/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.899783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b82f7-ac69-42d3-8f6b-26bab1c5ada4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eaf4b3186ab472da0cd67bf31fd5b03, user_id=c2fa14b07aa842f3b00cf1809ce676de, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:40.902530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b82f7-ac69-42d3-8f6b-26bab1c5ada4 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:40.919114 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1305/2614] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:40 2022] PUT /identity/v3/projects/5eaf4b3186ab472da0cd67bf31fd5b03/users/c2fa14b07aa842f3b00cf1809ce676de/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:40.919957 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b82f7-ac69-42d3-8f6b-26bab1c5ada4 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:40.937365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bea5d0fe6d34482b85ac668fe33fb73'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:40.944227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bea5d0fe6d34482b85ac668fe33fb73'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:40.944911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bea5d0fe6d34482b85ac668fe33fb73'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:40.954770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bea5d0fe6d34482b85ac668fe33fb73'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:40.959498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:40.969549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb914d77-c133-4751-a96d-6005807a5d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bea5d0fe6d34482b85ac668fe33fb73'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:40.976646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1310/2615] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:40 2022] DELETE /identity/v3/users/1bea5d0fe6d34482b85ac668fe33fb73 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:40.977998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c940c66-00c5-4916-b9c2-974256962d61 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:40.978347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c940c66-00c5-4916-b9c2-974256962d61 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:40.978734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c940c66-00c5-4916-b9c2-974256962d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:40.985187 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b82f7-ac69-42d3-8f6b-26bab1c5ada4 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.005607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-0c940c66-00c5-4916-b9c2-974256962d61 None None] MFA Rules not processed for user `c2fa14b07aa842f3b00cf1809ce676de`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:41.007060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0c940c66-00c5-4916-b9c2-974256962d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.042511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19ef76-117d-4bfe-bcae-6f954b20dc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.043239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19ef76-117d-4bfe-bcae-6f954b20dc2a admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.043552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19ef76-117d-4bfe-bcae-6f954b20dc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.043830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19ef76-117d-4bfe-bcae-6f954b20dc2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.044559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19ef76-117d-4bfe-bcae-6f954b20dc2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.046207 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19ef76-117d-4bfe-bcae-6f954b20dc2a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.056164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1306/2616] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:41.057418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d425f-0245-409e-8b4c-c9d888ca8c98 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.057790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d425f-0245-409e-8b4c-c9d888ca8c98 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.058014 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d425f-0245-409e-8b4c-c9d888ca8c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.072769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1311/2617] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:41.073616 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c940c66-00c5-4916-b9c2-974256962d61 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.077296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ec1d425f-0245-409e-8b4c-c9d888ca8c98 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:41.077414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0c940c66-00c5-4916-b9c2-974256962d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.078481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ec1d425f-0245-409e-8b4c-c9d888ca8c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.086049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1307/2618] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:41.086818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d425f-0245-409e-8b4c-c9d888ca8c98 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.088387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ec1d425f-0245-409e-8b4c-c9d888ca8c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.090308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c940c66-00c5-4916-b9c2-974256962d61 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.094846 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506986e8-1270-4259-98cb-cab976e8c495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.095988 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506986e8-1270-4259-98cb-cab976e8c495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.096269 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506986e8-1270-4259-98cb-cab976e8c495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.096542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506986e8-1270-4259-98cb-cab976e8c495 admin admin] PATH_INFO: `/v3/projects/af9dc5c08c5340bc8f14c156a4d068cb/users/3334bcb89f234aeba31090dce59fa440/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.100960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d425f-0245-409e-8b4c-c9d888ca8c98 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.105981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65f951-ac3c-4744-a345-1f6be3faca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.106646 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65f951-ac3c-4744-a345-1f6be3faca23 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.106929 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65f951-ac3c-4744-a345-1f6be3faca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.107208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65f951-ac3c-4744-a345-1f6be3faca23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.107962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65f951-ac3c-4744-a345-1f6be3faca23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.109871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65f951-ac3c-4744-a345-1f6be3faca23 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.110987 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506986e8-1270-4259-98cb-cab976e8c495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9dc5c08c5340bc8f14c156a4d068cb, user_id=3334bcb89f234aeba31090dce59fa440, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.113695 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506986e8-1270-4259-98cb-cab976e8c495 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.128984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1312/2619] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:41 2022] PUT /identity/v3/projects/af9dc5c08c5340bc8f14c156a4d068cb/users/3334bcb89f234aeba31090dce59fa440/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:41.129816 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506986e8-1270-4259-98cb-cab976e8c495 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.131394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-506986e8-1270-4259-98cb-cab976e8c495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.141617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1308/2620] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:41 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:41.142642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65f951-ac3c-4744-a345-1f6be3faca23 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.188370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:41.190079 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506986e8-1270-4259-98cb-cab976e8c495 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.208743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65f951-ac3c-4744-a345-1f6be3faca23 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.213516 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c0e6d-d6b5-41e8-a2f0-e63dc30bfee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.214235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c0e6d-d6b5-41e8-a2f0-e63dc30bfee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.214516 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c0e6d-d6b5-41e8-a2f0-e63dc30bfee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.214788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c0e6d-d6b5-41e8-a2f0-e63dc30bfee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.215545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c0e6d-d6b5-41e8-a2f0-e63dc30bfee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.217141 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c0e6d-d6b5-41e8-a2f0-e63dc30bfee4 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.227782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1309/2621] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:41.228571 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c0e6d-d6b5-41e8-a2f0-e63dc30bfee4 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.238694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c0e6d-d6b5-41e8-a2f0-e63dc30bfee4 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.238841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276c8fc-d5fc-4273-8979-7af915a062ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.239498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276c8fc-d5fc-4273-8979-7af915a062ae service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.239761 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276c8fc-d5fc-4273-8979-7af915a062ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.240036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276c8fc-d5fc-4273-8979-7af915a062ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.243528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d43142-06ab-48c6-a0af-aa547ee0e51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.244218 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d43142-06ab-48c6-a0af-aa547ee0e51c admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.244509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d43142-06ab-48c6-a0af-aa547ee0e51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.244817 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d43142-06ab-48c6-a0af-aa547ee0e51c admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.245573 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d43142-06ab-48c6-a0af-aa547ee0e51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.247554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d43142-06ab-48c6-a0af-aa547ee0e51c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.249983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276c8fc-d5fc-4273-8979-7af915a062ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.251449 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276c8fc-d5fc-4273-8979-7af915a062ae service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.261462 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1313/2622] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:41.262347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276c8fc-d5fc-4273-8979-7af915a062ae service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.271454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1310/2623] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:41 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:41.272044 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276c8fc-d5fc-4273-8979-7af915a062ae service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.275379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d43142-06ab-48c6-a0af-aa547ee0e51c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.277100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdbe0e-e7ed-4b12-9c42-1518d5fd544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.278087 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdbe0e-e7ed-4b12-9c42-1518d5fd544c admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.278399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdbe0e-e7ed-4b12-9c42-1518d5fd544c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.278881 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdbe0e-e7ed-4b12-9c42-1518d5fd544c admin admin] PATH_INFO: `/v3/projects/af9dc5c08c5340bc8f14c156a4d068cb/users/3334bcb89f234aeba31090dce59fa440/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.281003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdbe0e-e7ed-4b12-9c42-1518d5fd544c admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9dc5c08c5340bc8f14c156a4d068cb, user_id=3334bcb89f234aeba31090dce59fa440, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.283981 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdbe0e-e7ed-4b12-9c42-1518d5fd544c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.286411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d43142-06ab-48c6-a0af-aa547ee0e51c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.287942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:41.292986 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac651d40-92b3-4e9a-9add-d1aa2a986610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.293965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac651d40-92b3-4e9a-9add-d1aa2a986610 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.294424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac651d40-92b3-4e9a-9add-d1aa2a986610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.294710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac651d40-92b3-4e9a-9add-d1aa2a986610 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.295765 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac651d40-92b3-4e9a-9add-d1aa2a986610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.297510 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac651d40-92b3-4e9a-9add-d1aa2a986610 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.303330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1314/2624] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:41 2022] PUT /identity/v3/projects/af9dc5c08c5340bc8f14c156a4d068cb/users/3334bcb89f234aeba31090dce59fa440/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:41.308865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fa325-8efd-4003-b6c2-4441753bcb83 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.309187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fa325-8efd-4003-b6c2-4441753bcb83 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.309429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fa325-8efd-4003-b6c2-4441753bcb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.310133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1311/2625] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:41.313977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac651d40-92b3-4e9a-9add-d1aa2a986610 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.324661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac651d40-92b3-4e9a-9add-d1aa2a986610 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.341698 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-aa6fa325-8efd-4003-b6c2-4441753bcb83 None None] MFA Rules not processed for user `3334bcb89f234aeba31090dce59fa440`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:41.342928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-aa6fa325-8efd-4003-b6c2-4441753bcb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.390857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43562f4-c146-48a2-9b41-20db09f75499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.392073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43562f4-c146-48a2-9b41-20db09f75499 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.392355 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43562f4-c146-48a2-9b41-20db09f75499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.392872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43562f4-c146-48a2-9b41-20db09f75499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.393358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43562f4-c146-48a2-9b41-20db09f75499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.395086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43562f4-c146-48a2-9b41-20db09f75499 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.406047 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1312/2626] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:41.409941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43562f4-c146-48a2-9b41-20db09f75499 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.421122 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43562f4-c146-48a2-9b41-20db09f75499 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.429480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fd698-c5c0-4595-ba3f-fbb538c17724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.430448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fd698-c5c0-4595-ba3f-fbb538c17724 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.431267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fd698-c5c0-4595-ba3f-fbb538c17724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.431592 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fd698-c5c0-4595-ba3f-fbb538c17724 admin admin] PATH_INFO: `/v3/projects/c1f03404ac12487088812ab1a9941a68/users/7a216d37e89f447f83ffc7dffb068dea/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.441313 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1315/2627] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:41.450731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fd698-c5c0-4595-ba3f-fbb538c17724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f03404ac12487088812ab1a9941a68, user_id=7a216d37e89f447f83ffc7dffb068dea, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.453062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fa325-8efd-4003-b6c2-4441753bcb83 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.453829 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fd698-c5c0-4595-ba3f-fbb538c17724 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.454669 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-aa6fa325-8efd-4003-b6c2-4441753bcb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.469748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1313/2628] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:41 2022] PUT /identity/v3/projects/c1f03404ac12487088812ab1a9941a68/users/7a216d37e89f447f83ffc7dffb068dea/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:41.481190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f6f91-398a-4957-92a9-7699557314ab None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.481490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f6f91-398a-4957-92a9-7699557314ab None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.481747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f6f91-398a-4957-92a9-7699557314ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.511514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-fb3f6f91-398a-4957-92a9-7699557314ab None None] MFA Rules not processed for user `7a216d37e89f447f83ffc7dffb068dea`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:41.517308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fb3f6f91-398a-4957-92a9-7699557314ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.529270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fa325-8efd-4003-b6c2-4441753bcb83 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.585397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d6d0b-71f1-4c16-bf81-9e508f27b807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.586158 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6d0b-71f1-4c16-bf81-9e508f27b807 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.586539 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6d0b-71f1-4c16-bf81-9e508f27b807 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.586911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6d0b-71f1-4c16-bf81-9e508f27b807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.593064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1314/2629] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:41.597362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f6f91-398a-4957-92a9-7699557314ab None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.598749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fb3f6f91-398a-4957-92a9-7699557314ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.609525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:41.638984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:41.652680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d6d0b-71f1-4c16-bf81-9e508f27b807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.654139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d6d0b-71f1-4c16-bf81-9e508f27b807 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.665745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1316/2630] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:41.666570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d6d0b-71f1-4c16-bf81-9e508f27b807 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.666744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f6f91-398a-4957-92a9-7699557314ab None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.671437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.672316 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.672594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.672891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] PATH_INFO: `/v3/projects/d5f1a3e07ca24585bbb746b4a768d9cb` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.674242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5f1a3e07ca24585bbb746b4a768d9cb)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.676203 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.676624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d6d0b-71f1-4c16-bf81-9e508f27b807 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.681689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094e307-8646-49c1-bf2c-a1ef4c47c3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.682351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094e307-8646-49c1-bf2c-a1ef4c47c3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.683009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094e307-8646-49c1-bf2c-a1ef4c47c3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.683009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094e307-8646-49c1-bf2c-a1ef4c47c3f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.683681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094e307-8646-49c1-bf2c-a1ef4c47c3f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.685583 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094e307-8646-49c1-bf2c-a1ef4c47c3f1 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.715580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1317/2631] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:41 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:41.719878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094e307-8646-49c1-bf2c-a1ef4c47c3f1 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.735312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5f1a3e07ca24585bbb746b4a768d9cb', 'request_id': 'req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:41.744640 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5f1a3e07ca24585bbb746b4a768d9cb', 'request_id': 'req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:41.762250 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5f1a3e07ca24585bbb746b4a768d9cb', 'request_id': 'req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:41.781336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] The token cache is being invalidate because project d5f1a3e07ca24585bbb746b4a768d9cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:41.781728 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:41.782446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:41.784323 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1315/2632] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:41 2022] DELETE /identity/v3/projects/d5f1a3e07ca24585bbb746b4a768d9cb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:41.787295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.787295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.807069 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094e307-8646-49c1-bf2c-a1ef4c47c3f1 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.811863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e1a27-0ca5-4289-8b92-c9caf7e9e00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.812535 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e1a27-0ca5-4289-8b92-c9caf7e9e00a admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.812869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e1a27-0ca5-4289-8b92-c9caf7e9e00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.813064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e1a27-0ca5-4289-8b92-c9caf7e9e00a admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.813758 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e1a27-0ca5-4289-8b92-c9caf7e9e00a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.815815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e1a27-0ca5-4289-8b92-c9caf7e9e00a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.843529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1318/2633] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:41 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:41.844413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e1a27-0ca5-4289-8b92-c9caf7e9e00a admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.846063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b25e1a27-0ca5-4289-8b92-c9caf7e9e00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.853692 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e4b67-cf42-4904-92c9-d272b87e6ad7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.858331 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e1a27-0ca5-4289-8b92-c9caf7e9e00a admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:41.865712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.867008 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.867324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.867606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] PATH_INFO: `/v3/users/e06cdbc238644d80b48503bced82e0c6` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.868908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e06cdbc238644d80b48503bced82e0c6)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:41.871663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:41.908971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52fb70-c369-4e9f-b14a-d01f948a2cf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:41.909639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52fb70-c369-4e9f-b14a-d01f948a2cf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:41.909915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52fb70-c369-4e9f-b14a-d01f948a2cf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:41.910185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52fb70-c369-4e9f-b14a-d01f948a2cf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:41.912159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8c52fb70-c369-4e9f-b14a-d01f948a2cf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:41.925644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:41.939362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e06cdbc238644d80b48503bced82e0c6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:41.946719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e06cdbc238644d80b48503bced82e0c6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:41.947483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e06cdbc238644d80b48503bced82e0c6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:41.958031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e06cdbc238644d80b48503bced82e0c6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:41.976415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e06cdbc238644d80b48503bced82e0c6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:41.992139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1319/2634] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:41 2022] DELETE /identity/v3/users/e06cdbc238644d80b48503bced82e0c6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:41.992139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:41.992139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.027314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52fb70-c369-4e9f-b14a-d01f948a2cf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.029063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52fb70-c369-4e9f-b14a-d01f948a2cf5 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.045677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1316/2635] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:42.048261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:42.105463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c87536-106f-4e98-b7e0-b3b680a07afa admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.112709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811817b8-b1c7-4c7c-b575-4002a274b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.113830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811817b8-b1c7-4c7c-b575-4002a274b8cd admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.114288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811817b8-b1c7-4c7c-b575-4002a274b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.114982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811817b8-b1c7-4c7c-b575-4002a274b8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.117040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811817b8-b1c7-4c7c-b575-4002a274b8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.118377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811817b8-b1c7-4c7c-b575-4002a274b8cd admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.139735 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1320/2636] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:42.140932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52fb70-c369-4e9f-b14a-d01f948a2cf5 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.142556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8c52fb70-c369-4e9f-b14a-d01f948a2cf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.162197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52fb70-c369-4e9f-b14a-d01f948a2cf5 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.168222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbbe80-dda3-4e32-b5ab-1d66b6179b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.173221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbbe80-dda3-4e32-b5ab-1d66b6179b23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.173577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbbe80-dda3-4e32-b5ab-1d66b6179b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.173933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbbe80-dda3-4e32-b5ab-1d66b6179b23 admin admin] PATH_INFO: `/v3/projects/ab4f344d551d41fa8691d70c229c65ce/users/3a50cd44d7ff4b05803e8c7d2bf779a8/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.191422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbbe80-dda3-4e32-b5ab-1d66b6179b23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4f344d551d41fa8691d70c229c65ce, user_id=3a50cd44d7ff4b05803e8c7d2bf779a8, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.194671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbbe80-dda3-4e32-b5ab-1d66b6179b23 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.223811 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1317/2637] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:42 2022] PUT /identity/v3/projects/ab4f344d551d41fa8691d70c229c65ce/users/3a50cd44d7ff4b05803e8c7d2bf779a8/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:42.228007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811817b8-b1c7-4c7c-b575-4002a274b8cd admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.238501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811817b8-b1c7-4c7c-b575-4002a274b8cd admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.315059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6f460-ef71-45d2-8d6b-3c3c5fddddcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.316486 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f460-ef71-45d2-8d6b-3c3c5fddddcc admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.316773 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f460-ef71-45d2-8d6b-3c3c5fddddcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.317048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f460-ef71-45d2-8d6b-3c3c5fddddcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.317905 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6f460-ef71-45d2-8d6b-3c3c5fddddcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.319566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6f460-ef71-45d2-8d6b-3c3c5fddddcc admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.325934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbbe80-dda3-4e32-b5ab-1d66b6179b23 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.329070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ecdbbe80-dda3-4e32-b5ab-1d66b6179b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.336129 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1321/2638] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:42.337344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6f460-ef71-45d2-8d6b-3c3c5fddddcc admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.348980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6f460-ef71-45d2-8d6b-3c3c5fddddcc admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.351746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbbe80-dda3-4e32-b5ab-1d66b6179b23 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.355120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b3ff9-2f9e-4985-bc68-9b5c47322cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.356093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b3ff9-2f9e-4985-bc68-9b5c47322cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.356622 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b3ff9-2f9e-4985-bc68-9b5c47322cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.356909 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b3ff9-2f9e-4985-bc68-9b5c47322cfe admin admin] PATH_INFO: `/v3/projects/ab4f344d551d41fa8691d70c229c65ce/users/3a50cd44d7ff4b05803e8c7d2bf779a8/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.359026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b3ff9-2f9e-4985-bc68-9b5c47322cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4f344d551d41fa8691d70c229c65ce, user_id=3a50cd44d7ff4b05803e8c7d2bf779a8, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.360683 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.361781 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b3ff9-2f9e-4985-bc68-9b5c47322cfe admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.361910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.362330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.362622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] PATH_INFO: `/v3/projects/32eeff8b22a44703bf7b2dc973c9a0e4` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.364368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32eeff8b22a44703bf7b2dc973c9a0e4)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.366528 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.373874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1322/2639] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:42 2022] PUT /identity/v3/projects/ab4f344d551d41fa8691d70c229c65ce/users/3a50cd44d7ff4b05803e8c7d2bf779a8/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:42.382993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b3ff9-2f9e-4985-bc68-9b5c47322cfe admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.383956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9e8b3ff9-2f9e-4985-bc68-9b5c47322cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.419555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32eeff8b22a44703bf7b2dc973c9a0e4', 'request_id': 'req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:42.426143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32eeff8b22a44703bf7b2dc973c9a0e4', 'request_id': 'req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:42.432414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32eeff8b22a44703bf7b2dc973c9a0e4', 'request_id': 'req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:42.446992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] The token cache is being invalidate because project 32eeff8b22a44703bf7b2dc973c9a0e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:42.447300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:42.447933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c0e3b87b-5607-4e0e-8fd2-4f88913a33d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:42.449566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1318/2640] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:42 2022] DELETE /identity/v3/projects/32eeff8b22a44703bf7b2dc973c9a0e4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:42.451090 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce33e-376e-4fdb-9f8b-d51595ecbf40 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.451463 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce33e-376e-4fdb-9f8b-d51595ecbf40 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.451746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce33e-376e-4fdb-9f8b-d51595ecbf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.453436 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b3ff9-2f9e-4985-bc68-9b5c47322cfe admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.480643 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2ebce33e-376e-4fdb-9f8b-d51595ecbf40 None None] MFA Rules not processed for user `3a50cd44d7ff4b05803e8c7d2bf779a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:42.481879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2ebce33e-376e-4fdb-9f8b-d51595ecbf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.506899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:42.523145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.524084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.524404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.524677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] PATH_INFO: `/v3/users/efaa9f10ab4d41459c22bcb8cf960368` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.525899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=efaa9f10ab4d41459c22bcb8cf960368)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.527955 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.542752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:42.555647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1319/2641] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:42.562549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553472a-c7c9-4555-be1b-b823ce77c155 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.563456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553472a-c7c9-4555-be1b-b823ce77c155 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.564240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553472a-c7c9-4555-be1b-b823ce77c155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.578694 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efaa9f10ab4d41459c22bcb8cf960368'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:42.584287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a553472a-c7c9-4555-be1b-b823ce77c155 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:42.585491 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a553472a-c7c9-4555-be1b-b823ce77c155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.585904 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efaa9f10ab4d41459c22bcb8cf960368'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:42.586613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efaa9f10ab4d41459c22bcb8cf960368'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:42.593972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1320/2642] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:42.594754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553472a-c7c9-4555-be1b-b823ce77c155 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.595765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efaa9f10ab4d41459c22bcb8cf960368'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:42.596295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a553472a-c7c9-4555-be1b-b823ce77c155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.610073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-3b6ac494-4c74-4d55-87ae-7631e07bda7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efaa9f10ab4d41459c22bcb8cf960368'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:42.616637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1323/2643] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:42 2022] DELETE /identity/v3/users/efaa9f10ab4d41459c22bcb8cf960368 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:42.618032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f18db-754b-4048-b4ae-ab9f326fbc0c None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.618426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f18db-754b-4048-b4ae-ab9f326fbc0c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.618768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f18db-754b-4048-b4ae-ab9f326fbc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.637956 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-312f18db-754b-4048-b4ae-ab9f326fbc0c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:42.639310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-312f18db-754b-4048-b4ae-ab9f326fbc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.653070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553472a-c7c9-4555-be1b-b823ce77c155 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.705927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7300a-9814-4618-96e9-fe82bd12170a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.706594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7300a-9814-4618-96e9-fe82bd12170a service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.707236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7300a-9814-4618-96e9-fe82bd12170a service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.707540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7300a-9814-4618-96e9-fe82bd12170a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.709444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b6a7300a-9814-4618-96e9-fe82bd12170a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.712134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1324/2644] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:42.713497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb9286-ebb2-48be-afea-716f43d1e977 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.713974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb9286-ebb2-48be-afea-716f43d1e977 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.714308 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb9286-ebb2-48be-afea-716f43d1e977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.732705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-27eb9286-ebb2-48be-afea-716f43d1e977 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:42.733928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-27eb9286-ebb2-48be-afea-716f43d1e977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.742889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1325/2645] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:42.743717 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb9286-ebb2-48be-afea-716f43d1e977 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.757869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb9286-ebb2-48be-afea-716f43d1e977 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.762986 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcaef83-18fd-489c-bfce-76c6614e7f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.763685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcaef83-18fd-489c-bfce-76c6614e7f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.764036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcaef83-18fd-489c-bfce-76c6614e7f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.764312 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcaef83-18fd-489c-bfce-76c6614e7f35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.765034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcaef83-18fd-489c-bfce-76c6614e7f35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:42.765340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcaef83-18fd-489c-bfce-76c6614e7f35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.766985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcaef83-18fd-489c-bfce-76c6614e7f35 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.778784 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1326/2646] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:29:42.779735 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcaef83-18fd-489c-bfce-76c6614e7f35 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.781023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7300a-9814-4618-96e9-fe82bd12170a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.781479 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4dcaef83-18fd-489c-bfce-76c6614e7f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:42.782548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7300a-9814-4618-96e9-fe82bd12170a service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.792329 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1321/2647] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:42.793011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7300a-9814-4618-96e9-fe82bd12170a service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.803778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7300a-9814-4618-96e9-fe82bd12170a service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.809078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae31ee-46ae-448e-8ca9-ab3f142aee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.809800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae31ee-46ae-448e-8ca9-ab3f142aee32 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.810088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae31ee-46ae-448e-8ca9-ab3f142aee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.810363 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae31ee-46ae-448e-8ca9-ab3f142aee32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.811053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae31ee-46ae-448e-8ca9-ab3f142aee32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.813016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae31ee-46ae-448e-8ca9-ab3f142aee32 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.851414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1322/2648] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:42 2022] POST /identity/v3/projects => generated 409 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:42.853304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae31ee-46ae-448e-8ca9-ab3f142aee32 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.854127 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcaef83-18fd-489c-bfce-76c6614e7f35 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.860985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:42.903425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:42.912567 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bc3da-14b3-41f1-a989-e10c01149e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.913230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bc3da-14b3-41f1-a989-e10c01149e95 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.913514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bc3da-14b3-41f1-a989-e10c01149e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.913811 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bc3da-14b3-41f1-a989-e10c01149e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.933898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae31ee-46ae-448e-8ca9-ab3f142aee32 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.939756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abd2d7-f25e-42fb-9623-6175b479ab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.941917 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abd2d7-f25e-42fb-9623-6175b479ab21 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.942761 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abd2d7-f25e-42fb-9623-6175b479ab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.943035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abd2d7-f25e-42fb-9623-6175b479ab21 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.943921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abd2d7-f25e-42fb-9623-6175b479ab21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.945856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abd2d7-f25e-42fb-9623-6175b479ab21 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.971771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1323/2649] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:42 2022] POST /identity/v3/users => generated 305 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:42.975931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abd2d7-f25e-42fb-9623-6175b479ab21 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:42.984194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bc3da-14b3-41f1-a989-e10c01149e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.985958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bc3da-14b3-41f1-a989-e10c01149e95 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.986462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abd2d7-f25e-42fb-9623-6175b479ab21 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:42.991333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e1cba-c3fc-4329-a926-847765eae5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:42.992003 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1cba-c3fc-4329-a926-847765eae5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:42.992282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1cba-c3fc-4329-a926-847765eae5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:42.992635 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1cba-c3fc-4329-a926-847765eae5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:42.993276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e1cba-c3fc-4329-a926-847765eae5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:42.995154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e1cba-c3fc-4329-a926-847765eae5ea admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:42.996588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1327/2650] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:43.010379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1324/2651] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:43.011882 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bc3da-14b3-41f1-a989-e10c01149e95 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.024809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bc3da-14b3-41f1-a989-e10c01149e95 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:43.030912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa930e-30e9-4316-8465-b215c2132e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:43.032004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa930e-30e9-4316-8465-b215c2132e81 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.032285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa930e-30e9-4316-8465-b215c2132e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.032791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa930e-30e9-4316-8465-b215c2132e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.033784 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa930e-30e9-4316-8465-b215c2132e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:43.036320 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa930e-30e9-4316-8465-b215c2132e81 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:43.048952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1328/2652] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:43.054415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e1cba-c3fc-4329-a926-847765eae5ea admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.067111 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e1cba-c3fc-4329-a926-847765eae5ea admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:43.090697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4762b11-7209-4cbf-beca-dd3d9896dfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:43.091971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4762b11-7209-4cbf-beca-dd3d9896dfcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.092421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4762b11-7209-4cbf-beca-dd3d9896dfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.092834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4762b11-7209-4cbf-beca-dd3d9896dfcc admin admin] PATH_INFO: `/v3/projects/32ff96cbd85448b48f56443d9c817956/users/0c29b916cdc246e2a7fd0fafb95e6f6d/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.126187 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4762b11-7209-4cbf-beca-dd3d9896dfcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ff96cbd85448b48f56443d9c817956, user_id=0c29b916cdc246e2a7fd0fafb95e6f6d, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:43.130832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4762b11-7209-4cbf-beca-dd3d9896dfcc admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:43.159290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1325/2653] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:43 2022] PUT /identity/v3/projects/32ff96cbd85448b48f56443d9c817956/users/0c29b916cdc246e2a7fd0fafb95e6f6d/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:43.168252 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7082e1-3f3e-494f-808b-976e2fc7b67b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.168856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7082e1-3f3e-494f-808b-976e2fc7b67b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.169358 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7082e1-3f3e-494f-808b-976e2fc7b67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.204716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-bf7082e1-3f3e-494f-808b-976e2fc7b67b None None] MFA Rules not processed for user `0c29b916cdc246e2a7fd0fafb95e6f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:43.206319 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-bf7082e1-3f3e-494f-808b-976e2fc7b67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:43.275653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1329/2654] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:43.280395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4762b11-7209-4cbf-beca-dd3d9896dfcc admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.321564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7082e1-3f3e-494f-808b-976e2fc7b67b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.323134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-bf7082e1-3f3e-494f-808b-976e2fc7b67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:43.351733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4762b11-7209-4cbf-beca-dd3d9896dfcc admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:43.358495 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d4eef-a45f-4176-99a3-d5571ff7a636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:43.359718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d4eef-a45f-4176-99a3-d5571ff7a636 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.360008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d4eef-a45f-4176-99a3-d5571ff7a636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.360286 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d4eef-a45f-4176-99a3-d5571ff7a636 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.361000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d4eef-a45f-4176-99a3-d5571ff7a636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:43.362926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d4eef-a45f-4176-99a3-d5571ff7a636 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:43.379975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:43.396245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1326/2655] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:43 2022] POST /identity/v3/projects => generated 409 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:43.397211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d4eef-a45f-4176-99a3-d5571ff7a636 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.404636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7082e1-3f3e-494f-808b-976e2fc7b67b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:43.459491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:43.460372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.460650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.460746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d4eef-a45f-4176-99a3-d5571ff7a636 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:43.460918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] PATH_INFO: `/v3/projects/654d19afeb8a468199281ad46f9217cb` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.462069 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654d19afeb8a468199281ad46f9217cb)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:43.464052 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:43.465395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a254bc0-2490-4b34-8dc7-b9537dc0489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:43.466069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a254bc0-2490-4b34-8dc7-b9537dc0489d admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.466356 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a254bc0-2490-4b34-8dc7-b9537dc0489d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.466626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a254bc0-2490-4b34-8dc7-b9537dc0489d admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.467947 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a254bc0-2490-4b34-8dc7-b9537dc0489d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:43.469879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a254bc0-2490-4b34-8dc7-b9537dc0489d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:43.498779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1327/2656] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:43 2022] POST /identity/v3/users => generated 311 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:43.500388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a254bc0-2490-4b34-8dc7-b9537dc0489d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.507834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:43.511859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a254bc0-2490-4b34-8dc7-b9537dc0489d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:43.523679 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654d19afeb8a468199281ad46f9217cb', 'request_id': 'req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:43.530367 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654d19afeb8a468199281ad46f9217cb', 'request_id': 'req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:43.536180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654d19afeb8a468199281ad46f9217cb', 'request_id': 'req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:43.552033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] The token cache is being invalidate because project 654d19afeb8a468199281ad46f9217cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:43.552344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:43.552978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:43.554576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1330/2657] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:43 2022] DELETE /identity/v3/projects/654d19afeb8a468199281ad46f9217cb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:43.558428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.559802 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:43.575212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5072c-66fa-43fc-a57d-16a33b13e62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:43.575868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5072c-66fa-43fc-a57d-16a33b13e62f admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.576154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5072c-66fa-43fc-a57d-16a33b13e62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.576433 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5072c-66fa-43fc-a57d-16a33b13e62f admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.577200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5072c-66fa-43fc-a57d-16a33b13e62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:43.578863 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5072c-66fa-43fc-a57d-16a33b13e62f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:43.589421 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1328/2658] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:43.593648 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5072c-66fa-43fc-a57d-16a33b13e62f admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.595415 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-09b5072c-66fa-43fc-a57d-16a33b13e62f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:43.607946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5072c-66fa-43fc-a57d-16a33b13e62f admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:43.612612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fc214-21d3-4171-b527-8cc9d5df42e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:43.613595 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fc214-21d3-4171-b527-8cc9d5df42e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.613881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fc214-21d3-4171-b527-8cc9d5df42e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.614156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fc214-21d3-4171-b527-8cc9d5df42e5 admin admin] PATH_INFO: `/v3/projects/e49762372ee34da6933b73000dd8389f/users/ec16e6db4233455cb4a8a6ad5d74fda4/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.624484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df13f6d-dbdb-4294-ba8a-d8f33bf4aa71 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:43.629422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fc214-21d3-4171-b527-8cc9d5df42e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e49762372ee34da6933b73000dd8389f, user_id=ec16e6db4233455cb4a8a6ad5d74fda4, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:43.630481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:43.631413 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.631697 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.631971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] PATH_INFO: `/v3/users/3334bcb89f234aeba31090dce59fa440` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.632120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fc214-21d3-4171-b527-8cc9d5df42e5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:43.633196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3334bcb89f234aeba31090dce59fa440)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:43.635157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:43.646880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1329/2659] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:43 2022] PUT /identity/v3/projects/e49762372ee34da6933b73000dd8389f/users/ec16e6db4233455cb4a8a6ad5d74fda4/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:43.650892 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fc214-21d3-4171-b527-8cc9d5df42e5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.652401 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:43.687230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3334bcb89f234aeba31090dce59fa440'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:43.695187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3334bcb89f234aeba31090dce59fa440'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:43.695947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3334bcb89f234aeba31090dce59fa440'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:43.704862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3334bcb89f234aeba31090dce59fa440'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:43.714722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fc214-21d3-4171-b527-8cc9d5df42e5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:43.718210 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3334bcb89f234aeba31090dce59fa440'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:43.719741 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80be7c-48c5-4ea0-81e2-709bf2e21aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:43.720697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80be7c-48c5-4ea0-81e2-709bf2e21aac admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.720975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80be7c-48c5-4ea0-81e2-709bf2e21aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.721245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80be7c-48c5-4ea0-81e2-709bf2e21aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.721964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80be7c-48c5-4ea0-81e2-709bf2e21aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:43.723759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80be7c-48c5-4ea0-81e2-709bf2e21aac admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:43.724565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1331/2660] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:43 2022] DELETE /identity/v3/users/3334bcb89f234aeba31090dce59fa440 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:29:43.739279 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1330/2661] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:43.741568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.743016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:43.811663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f6b9c-6743-470b-bcad-670098d8cee5 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:43.816354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0bf93-ff67-430e-a426-e1bedcd6a2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:43.817354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0bf93-ff67-430e-a426-e1bedcd6a2e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.817543 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0bf93-ff67-430e-a426-e1bedcd6a2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.817780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0bf93-ff67-430e-a426-e1bedcd6a2e6 admin admin] PATH_INFO: `/v3/projects/e49762372ee34da6933b73000dd8389f/users/ec16e6db4233455cb4a8a6ad5d74fda4/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.819885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0bf93-ff67-430e-a426-e1bedcd6a2e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e49762372ee34da6933b73000dd8389f, user_id=ec16e6db4233455cb4a8a6ad5d74fda4, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:43.822532 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0bf93-ff67-430e-a426-e1bedcd6a2e6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:43.833666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1332/2662] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:43 2022] PUT /identity/v3/projects/e49762372ee34da6933b73000dd8389f/users/ec16e6db4233455cb4a8a6ad5d74fda4/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:43.845256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444663f-401d-4470-974d-981c8dbcd235 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:43.845790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444663f-401d-4470-974d-981c8dbcd235 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:43.845946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444663f-401d-4470-974d-981c8dbcd235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:43.875120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-7444663f-401d-4470-974d-981c8dbcd235 None None] MFA Rules not processed for user `ec16e6db4233455cb4a8a6ad5d74fda4`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:43.879903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7444663f-401d-4470-974d-981c8dbcd235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:43.955971 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:43.962377 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1331/2663] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:43 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:43.973680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0bf93-ff67-430e-a426-e1bedcd6a2e6 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:43.976672 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-75f0bf93-ff67-430e-a426-e1bedcd6a2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:44.036992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444663f-401d-4470-974d-981c8dbcd235 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:44.039013 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7444663f-401d-4470-974d-981c8dbcd235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:44.059114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0bf93-ff67-430e-a426-e1bedcd6a2e6 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:44.063445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aaf544-0b7a-48ce-a4b7-4e0b9097abf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:44.064288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaf544-0b7a-48ce-a4b7-4e0b9097abf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.064624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaf544-0b7a-48ce-a4b7-4e0b9097abf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.075658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaf544-0b7a-48ce-a4b7-4e0b9097abf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.093600 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aaf544-0b7a-48ce-a4b7-4e0b9097abf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:44.095164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aaf544-0b7a-48ce-a4b7-4e0b9097abf3 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:44.108499 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1333/2664] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:43 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:44.138857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444663f-401d-4470-974d-981c8dbcd235 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:44.143520 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:44.144399 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.144681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.144951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] PATH_INFO: `/v3/projects/af9dc5c08c5340bc8f14c156a4d068cb` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.146130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af9dc5c08c5340bc8f14c156a4d068cb)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:44.148088 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:44.208557 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9dc5c08c5340bc8f14c156a4d068cb', 'request_id': 'req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:44.215015 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9dc5c08c5340bc8f14c156a4d068cb', 'request_id': 'req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:44.221358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9dc5c08c5340bc8f14c156a4d068cb', 'request_id': 'req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:44.234797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] The token cache is being invalidate because project af9dc5c08c5340bc8f14c156a4d068cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:44.235161 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:44.235826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:44.237390 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1332/2665] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:44 2022] DELETE /identity/v3/projects/af9dc5c08c5340bc8f14c156a4d068cb => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:44.243476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addbddd-2d5f-463f-bf85-cc81bc6c8793 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.243759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addbddd-2d5f-463f-bf85-cc81bc6c8793 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.244409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addbddd-2d5f-463f-bf85-cc81bc6c8793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.263573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-1addbddd-2d5f-463f-bf85-cc81bc6c8793 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:44.264848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1addbddd-2d5f-463f-bf85-cc81bc6c8793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:44.272500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:44.273981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:44.291669 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:44.328222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1334/2666] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:44.334229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b94c1e-cad8-4316-8ba9-255ac3ce8ec9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:44.336963 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca2de3-5e2f-433c-9617-8f9a65723262 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.337296 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca2de3-5e2f-433c-9617-8f9a65723262 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.337571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca2de3-5e2f-433c-9617-8f9a65723262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.338089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576f36b-7a3c-4632-a5cf-de98636f3dc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:44.338801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576f36b-7a3c-4632-a5cf-de98636f3dc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.339077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576f36b-7a3c-4632-a5cf-de98636f3dc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.339379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576f36b-7a3c-4632-a5cf-de98636f3dc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.341195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7576f36b-7a3c-4632-a5cf-de98636f3dc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:44.355442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-57ca2de3-5e2f-433c-9617-8f9a65723262 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:44.356641 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-57ca2de3-5e2f-433c-9617-8f9a65723262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:44.364544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1335/2667] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:44.368633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca2de3-5e2f-433c-9617-8f9a65723262 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:44.377831 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca2de3-5e2f-433c-9617-8f9a65723262 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:44.383124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424db53-a78a-4ed1-84a8-2dd665c99957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:44.383682 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:44.383809 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424db53-a78a-4ed1-84a8-2dd665c99957 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.384072 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424db53-a78a-4ed1-84a8-2dd665c99957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.384346 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424db53-a78a-4ed1-84a8-2dd665c99957 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.385070 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424db53-a78a-4ed1-84a8-2dd665c99957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:44.385398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424db53-a78a-4ed1-84a8-2dd665c99957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:44.387154 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424db53-a78a-4ed1-84a8-2dd665c99957 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:44.399133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1336/2668] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 06:29:44.402243 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576f36b-7a3c-4632-a5cf-de98636f3dc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:44.403421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424db53-a78a-4ed1-84a8-2dd665c99957 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:44.403834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576f36b-7a3c-4632-a5cf-de98636f3dc9 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:44.412915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424db53-a78a-4ed1-84a8-2dd665c99957 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:44.413456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1333/2669] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:44 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 06:29:44.417702 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80f079-5269-4150-9336-6008efede817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:44.418348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80f079-5269-4150-9336-6008efede817 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.418626 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80f079-5269-4150-9336-6008efede817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.418903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80f079-5269-4150-9336-6008efede817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.419619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80f079-5269-4150-9336-6008efede817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:44.421521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80f079-5269-4150-9336-6008efede817 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:44.448607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1337/2670] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:44 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:44.453085 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576f36b-7a3c-4632-a5cf-de98636f3dc9 service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:44.464342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576f36b-7a3c-4632-a5cf-de98636f3dc9 service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:44.529884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf956-333a-436b-bbeb-9bb5b48fe879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:44.530606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf956-333a-436b-bbeb-9bb5b48fe879 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.530878 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf956-333a-436b-bbeb-9bb5b48fe879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.531144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf956-333a-436b-bbeb-9bb5b48fe879 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.531875 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babf956-333a-436b-bbeb-9bb5b48fe879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:44.533797 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babf956-333a-436b-bbeb-9bb5b48fe879 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:44.558069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1334/2671] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:44 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:44.559566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80f079-5269-4150-9336-6008efede817 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:44.571488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80f079-5269-4150-9336-6008efede817 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:44.577708 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c110665-8832-4d01-8744-905c1b0b1b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:44.578383 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c110665-8832-4d01-8744-905c1b0b1b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.578666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c110665-8832-4d01-8744-905c1b0b1b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.578951 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c110665-8832-4d01-8744-905c1b0b1b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.579711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c110665-8832-4d01-8744-905c1b0b1b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:44.581335 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c110665-8832-4d01-8744-905c1b0b1b99 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:44.592045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1338/2672] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:44.596159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf956-333a-436b-bbeb-9bb5b48fe879 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:44.606599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf956-333a-436b-bbeb-9bb5b48fe879 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:44.611338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338958c-15ed-43e4-8ada-205519f32050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:44.612041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338958c-15ed-43e4-8ada-205519f32050 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.612325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338958c-15ed-43e4-8ada-205519f32050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.612602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338958c-15ed-43e4-8ada-205519f32050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.613369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3338958c-15ed-43e4-8ada-205519f32050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:44.614979 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3338958c-15ed-43e4-8ada-205519f32050 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:44.626261 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1335/2673] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:29:44.631330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c110665-8832-4d01-8744-905c1b0b1b99 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:44.642085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c110665-8832-4d01-8744-905c1b0b1b99 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:44.647087 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691211a-963a-4a9c-8b10-98ea840a89e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:44.648012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691211a-963a-4a9c-8b10-98ea840a89e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.648323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691211a-963a-4a9c-8b10-98ea840a89e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.649417 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691211a-963a-4a9c-8b10-98ea840a89e2 admin admin] PATH_INFO: `/v3/projects/91f52d9454aa497c81b97529ca88e80f/users/68fcd9ee9b7f40b7b5387b67935e4707/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.667955 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691211a-963a-4a9c-8b10-98ea840a89e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f52d9454aa497c81b97529ca88e80f, user_id=68fcd9ee9b7f40b7b5387b67935e4707, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:44.669633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691211a-963a-4a9c-8b10-98ea840a89e2 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:44.685876 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1339/2674] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:44 2022] PUT /identity/v3/projects/91f52d9454aa497c81b97529ca88e80f/users/68fcd9ee9b7f40b7b5387b67935e4707/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:44.696287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895005dd-bc24-47f4-99a9-390d98c94bc3 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.696777 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895005dd-bc24-47f4-99a9-390d98c94bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.697398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895005dd-bc24-47f4-99a9-390d98c94bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.731209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-895005dd-bc24-47f4-99a9-390d98c94bc3 None None] MFA Rules not processed for user `68fcd9ee9b7f40b7b5387b67935e4707`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:44.732431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-895005dd-bc24-47f4-99a9-390d98c94bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:44.801370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1336/2675] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:44.824637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691211a-963a-4a9c-8b10-98ea840a89e2 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:44.890986 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691211a-963a-4a9c-8b10-98ea840a89e2 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:44.895674 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a55d0-d3c4-46c6-a0a3-c5a5f48da29d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:44.896324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a55d0-d3c4-46c6-a0a3-c5a5f48da29d service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:44.896601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a55d0-d3c4-46c6-a0a3-c5a5f48da29d service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:44.896873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a55d0-d3c4-46c6-a0a3-c5a5f48da29d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:44.906114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a55d0-d3c4-46c6-a0a3-c5a5f48da29d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:44.907633 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a55d0-d3c4-46c6-a0a3-c5a5f48da29d service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:44.916670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1340/2676] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:44.936634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895005dd-bc24-47f4-99a9-390d98c94bc3 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:44.988084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:45.002247 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895005dd-bc24-47f4-99a9-390d98c94bc3 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:45.009019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:45.009913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:45.010232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:45.010511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] PATH_INFO: `/v3/users/68fcd9ee9b7f40b7b5387b67935e4707` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:45.011852 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68fcd9ee9b7f40b7b5387b67935e4707)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:45.013840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:45.065975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68fcd9ee9b7f40b7b5387b67935e4707'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:45.073547 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68fcd9ee9b7f40b7b5387b67935e4707'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:45.082277 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68fcd9ee9b7f40b7b5387b67935e4707'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:45.082923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68fcd9ee9b7f40b7b5387b67935e4707'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:45.083708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68fcd9ee9b7f40b7b5387b67935e4707'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:45.102376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1337/2677] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:29:44 2022] DELETE /identity/v3/users/68fcd9ee9b7f40b7b5387b67935e4707 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:45.107004 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ab25a-f886-49b0-9b4e-dca6dd6e3588 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:45.107507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ab25a-f886-49b0-9b4e-dca6dd6e3588 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:45.107764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ab25a-f886-49b0-9b4e-dca6dd6e3588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:45.125475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-b38ab25a-f886-49b0-9b4e-dca6dd6e3588 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:45.126684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b38ab25a-f886-49b0-9b4e-dca6dd6e3588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:45.178105 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:45.192942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1341/2678] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:29:45.205637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:45.207407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:45.270746 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e6377-2c30-458d-a012-6a77eaba85b3 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:45.274598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d4497-9c3e-4288-92eb-7c9ec8214917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:45.275269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d4497-9c3e-4288-92eb-7c9ec8214917 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:45.275555 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d4497-9c3e-4288-92eb-7c9ec8214917 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:45.275826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d4497-9c3e-4288-92eb-7c9ec8214917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:45.285153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d4497-9c3e-4288-92eb-7c9ec8214917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:45.286583 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d4497-9c3e-4288-92eb-7c9ec8214917 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:45.295968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1338/2679] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:45.580168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ab25a-f886-49b0-9b4e-dca6dd6e3588 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:45.581737 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b38ab25a-f886-49b0-9b4e-dca6dd6e3588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:45.594524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ab25a-f886-49b0-9b4e-dca6dd6e3588 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:45.599602 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:45.600485 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:45.600765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:45.601042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] PATH_INFO: `/v3/projects/91f52d9454aa497c81b97529ca88e80f` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:45.602314 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f52d9454aa497c81b97529ca88e80f)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:45.604376 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:45.653024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f52d9454aa497c81b97529ca88e80f', 'request_id': 'req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:45.659034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f52d9454aa497c81b97529ca88e80f', 'request_id': 'req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:45.664968 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f52d9454aa497c81b97529ca88e80f', 'request_id': 'req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:45.679759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] The token cache is being invalidate because project 91f52d9454aa497c81b97529ca88e80f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:29:45.680040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:45.680653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ca1ee2dc-d22c-40ed-83fa-8ba9b196bd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:29:45.682277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1342/2680] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:29:45 2022] DELETE /identity/v3/projects/91f52d9454aa497c81b97529ca88e80f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:45.687217 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4f9f0-3de7-437a-aa6e-df0e4d741354 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:45.687545 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4f9f0-3de7-437a-aa6e-df0e4d741354 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:45.687822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4f9f0-3de7-437a-aa6e-df0e4d741354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:45.710288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-49a4f9f0-3de7-437a-aa6e-df0e4d741354 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:45.712319 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-49a4f9f0-3de7-437a-aa6e-df0e4d741354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:45.779985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1339/2681] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:45.788043 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ba9e5-94ad-49d9-a277-9619be0ffa78 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:45.788332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ba9e5-94ad-49d9-a277-9619be0ffa78 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:45.788620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ba9e5-94ad-49d9-a277-9619be0ffa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:45.808441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d92ba9e5-94ad-49d9-a277-9619be0ffa78 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:45.809650 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d92ba9e5-94ad-49d9-a277-9619be0ffa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:45.818911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1343/2682] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:45.823639 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4f9f0-3de7-437a-aa6e-df0e4d741354 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:45.838703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4f9f0-3de7-437a-aa6e-df0e4d741354 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:45.847996 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04846bb5-b7df-4337-a40a-fffcfe1ee6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:45.848926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04846bb5-b7df-4337-a40a-fffcfe1ee6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:45.849238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04846bb5-b7df-4337-a40a-fffcfe1ee6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:45.849535 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04846bb5-b7df-4337-a40a-fffcfe1ee6f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:45.850589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04846bb5-b7df-4337-a40a-fffcfe1ee6f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:29:45.850953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04846bb5-b7df-4337-a40a-fffcfe1ee6f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:45.853325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04846bb5-b7df-4337-a40a-fffcfe1ee6f7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:45.862588 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:45.866835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1340/2683] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:29:45.872310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ba9e5-94ad-49d9-a277-9619be0ffa78 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:45.885866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ba9e5-94ad-49d9-a277-9619be0ffa78 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:45.891280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c98a2e-68c7-4630-b41a-85531c94ba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:45.891935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c98a2e-68c7-4630-b41a-85531c94ba23 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:45.892213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c98a2e-68c7-4630-b41a-85531c94ba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:45.892522 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c98a2e-68c7-4630-b41a-85531c94ba23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:45.893271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c98a2e-68c7-4630-b41a-85531c94ba23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:45.895178 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c98a2e-68c7-4630-b41a-85531c94ba23 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:45.924639 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1344/2684] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:29:45 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:29:45.929499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04846bb5-b7df-4337-a40a-fffcfe1ee6f7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:45.944949 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04846bb5-b7df-4337-a40a-fffcfe1ee6f7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:46.005690 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba404279-2493-4703-b04b-4e346b0d4c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:46.006359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404279-2493-4703-b04b-4e346b0d4c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:46.006642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404279-2493-4703-b04b-4e346b0d4c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:46.006914 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404279-2493-4703-b04b-4e346b0d4c33 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:46.007655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba404279-2493-4703-b04b-4e346b0d4c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:46.009584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba404279-2493-4703-b04b-4e346b0d4c33 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:46.033484 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1341/2685] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:29:45 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:29:46.037777 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c98a2e-68c7-4630-b41a-85531c94ba23 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:46.048521 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c98a2e-68c7-4630-b41a-85531c94ba23 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:46.053546 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da794bbd-3f11-4737-a5db-7699ecb13fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:46.054205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da794bbd-3f11-4737-a5db-7699ecb13fec admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:46.054488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da794bbd-3f11-4737-a5db-7699ecb13fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:46.054759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da794bbd-3f11-4737-a5db-7699ecb13fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:46.055541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da794bbd-3f11-4737-a5db-7699ecb13fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:46.057151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da794bbd-3f11-4737-a5db-7699ecb13fec admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:46.068481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1345/2686] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:46.072903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba404279-2493-4703-b04b-4e346b0d4c33 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:46.088830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba404279-2493-4703-b04b-4e346b0d4c33 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:46.094687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bd5b8-18c9-401f-a921-4e6d07fead97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:46.095384 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bd5b8-18c9-401f-a921-4e6d07fead97 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:46.095673 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bd5b8-18c9-401f-a921-4e6d07fead97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:46.095946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bd5b8-18c9-401f-a921-4e6d07fead97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:46.096706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bd5b8-18c9-401f-a921-4e6d07fead97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:46.098306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bd5b8-18c9-401f-a921-4e6d07fead97 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:46.108876 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1342/2687] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:29:46.112879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da794bbd-3f11-4737-a5db-7699ecb13fec admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:46.123421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da794bbd-3f11-4737-a5db-7699ecb13fec admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:46.129608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e013dd-3629-4470-825d-0595095127e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:46.130513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e013dd-3629-4470-825d-0595095127e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:46.130796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e013dd-3629-4470-825d-0595095127e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:46.131069 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e013dd-3629-4470-825d-0595095127e9 admin admin] PATH_INFO: `/v3/projects/8cfe127799e0444d959228abe7ffe5e1/users/c1785d0b419643aa90fd3e9146a5c6c6/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:46.149656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e013dd-3629-4470-825d-0595095127e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cfe127799e0444d959228abe7ffe5e1, user_id=c1785d0b419643aa90fd3e9146a5c6c6, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:46.152489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e013dd-3629-4470-825d-0595095127e9 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:46.168475 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1346/2688] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:29:46 2022] PUT /identity/v3/projects/8cfe127799e0444d959228abe7ffe5e1/users/c1785d0b419643aa90fd3e9146a5c6c6/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:29:46.179840 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12783f94-1f5e-4d6b-814c-92794e7b2811 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:46.180134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12783f94-1f5e-4d6b-814c-92794e7b2811 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:46.180431 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12783f94-1f5e-4d6b-814c-92794e7b2811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:46.207549 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-12783f94-1f5e-4d6b-814c-92794e7b2811 None None] MFA Rules not processed for user `c1785d0b419643aa90fd3e9146a5c6c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:46.208762 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-12783f94-1f5e-4d6b-814c-92794e7b2811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:46.276809 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1343/2689] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:46.288424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e013dd-3629-4470-825d-0595095127e9 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:46.290415 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-29e013dd-3629-4470-825d-0595095127e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:46.352762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e013dd-3629-4470-825d-0595095127e9 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:46.357278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7346a-9057-4e3a-b339-89e254ecf575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:46.357933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7346a-9057-4e3a-b339-89e254ecf575 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:46.358218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7346a-9057-4e3a-b339-89e254ecf575 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:46.358491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7346a-9057-4e3a-b339-89e254ecf575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:46.368132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7346a-9057-4e3a-b339-89e254ecf575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:46.370349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7346a-9057-4e3a-b339-89e254ecf575 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:46.384564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1347/2690] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:50.670650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12783f94-1f5e-4d6b-814c-92794e7b2811 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:50.680969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12783f94-1f5e-4d6b-814c-92794e7b2811 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:50.684789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cee075-d172-4389-bdb5-eedd2f626744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:50.685449 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cee075-d172-4389-bdb5-eedd2f626744 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:50.685725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cee075-d172-4389-bdb5-eedd2f626744 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:50.685977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cee075-d172-4389-bdb5-eedd2f626744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:50.687946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e1cee075-d172-4389-bdb5-eedd2f626744 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:50.755870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cee075-d172-4389-bdb5-eedd2f626744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:50.757247 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cee075-d172-4389-bdb5-eedd2f626744 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:50.766536 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1344/2691] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:52.612914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7346a-9057-4e3a-b339-89e254ecf575 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:52.622124 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7346a-9057-4e3a-b339-89e254ecf575 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:52.626540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3608c-e623-45a9-8cc5-bff3e7832a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:52.627186 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3608c-e623-45a9-8cc5-bff3e7832a6c service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:52.627504 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3608c-e623-45a9-8cc5-bff3e7832a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:52.627782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3608c-e623-45a9-8cc5-bff3e7832a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:52.630031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e6f3608c-e623-45a9-8cc5-bff3e7832a6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:52.648709 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:52.698006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3608c-e623-45a9-8cc5-bff3e7832a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:52.699494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3608c-e623-45a9-8cc5-bff3e7832a6c service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:52.712612 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1348/2692] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:29:53.035788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdc46a-3481-4c6a-adaf-3ca3b6e745fb None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:53.036296 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdc46a-3481-4c6a-adaf-3ca3b6e745fb None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:53.036387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdc46a-3481-4c6a-adaf-3ca3b6e745fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:53.049976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:29:53.058774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-fbbdc46a-3481-4c6a-adaf-3ca3b6e745fb None None] MFA Rules not processed for user `c1785d0b419643aa90fd3e9146a5c6c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:29:53.061253 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fbbdc46a-3481-4c6a-adaf-3ca3b6e745fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:53.068579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1345/2693] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:29:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:29:53.561023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3608c-e623-45a9-8cc5-bff3e7832a6c service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:29:53.562917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e6f3608c-e623-45a9-8cc5-bff3e7832a6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:53.655406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3608c-e623-45a9-8cc5-bff3e7832a6c service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:29:53.664785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ca6d6-3cca-4796-8f6b-46d5b7d67a6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:29:53.665873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ca6d6-3cca-4796-8f6b-46d5b7d67a6b service placement] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:29:53.666382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ca6d6-3cca-4796-8f6b-46d5b7d67a6b service placement] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:29:53.666836 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ca6d6-3cca-4796-8f6b-46d5b7d67a6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:29:53.673071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6e8ca6d6-3cca-4796-8f6b-46d5b7d67a6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:29:53.687053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ca6d6-3cca-4796-8f6b-46d5b7d67a6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:29:53.688955 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ca6d6-3cca-4796-8f6b-46d5b7d67a6b service placement] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:29:53.700592 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1349/2694] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:29:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:01.269263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdc46a-3481-4c6a-adaf-3ca3b6e745fb None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:01.270697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fbbdc46a-3481-4c6a-adaf-3ca3b6e745fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:01.337669 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdc46a-3481-4c6a-adaf-3ca3b6e745fb None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:01.344069 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:01.344897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:01.345261 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:01.345538 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] PATH_INFO: `/v3/users/0c29b916cdc246e2a7fd0fafb95e6f6d` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:01.346977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c29b916cdc246e2a7fd0fafb95e6f6d)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:01.348968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:01.399934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c29b916cdc246e2a7fd0fafb95e6f6d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:01.406753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c29b916cdc246e2a7fd0fafb95e6f6d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:01.416064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c29b916cdc246e2a7fd0fafb95e6f6d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:01.416759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c29b916cdc246e2a7fd0fafb95e6f6d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:01.417404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c29b916cdc246e2a7fd0fafb95e6f6d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:01.435823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1346/2695] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:01 2022] DELETE /identity/v3/users/0c29b916cdc246e2a7fd0fafb95e6f6d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:01.440512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3caa0-3fa6-44db-b2cd-e1367d1210cd None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:01.441000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3caa0-3fa6-44db-b2cd-e1367d1210cd None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:01.441089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3caa0-3fa6-44db-b2cd-e1367d1210cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:01.458937 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-a6b3caa0-3fa6-44db-b2cd-e1367d1210cd None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:01.460168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a6b3caa0-3fa6-44db-b2cd-e1367d1210cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:01.539975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1350/2696] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:01.551789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:01.552818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:01.619850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d88fb-584a-4688-bbf9-5559ca26bd7c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:01.623725 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7816a2-2478-4887-9ac0-133b15c9f913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:01.624456 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7816a2-2478-4887-9ac0-133b15c9f913 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:01.624742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7816a2-2478-4887-9ac0-133b15c9f913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:01.625018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7816a2-2478-4887-9ac0-133b15c9f913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:01.634901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7816a2-2478-4887-9ac0-133b15c9f913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:01.636435 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7816a2-2478-4887-9ac0-133b15c9f913 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:01.645932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1347/2697] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 06:30:01.910398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3caa0-3fa6-44db-b2cd-e1367d1210cd None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:01.911899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a6b3caa0-3fa6-44db-b2cd-e1367d1210cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:01.932684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3caa0-3fa6-44db-b2cd-e1367d1210cd None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:01.938491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:01.939407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:01.939686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:01.939961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] PATH_INFO: `/v3/projects/32ff96cbd85448b48f56443d9c817956` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:01.941266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ff96cbd85448b48f56443d9c817956)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:01.943197 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:01.985500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:01.992974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ff96cbd85448b48f56443d9c817956', 'request_id': 'req-a1dc7bf9-c22f-4b60-8051-153165a35bc3'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:01.999055 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ff96cbd85448b48f56443d9c817956', 'request_id': 'req-a1dc7bf9-c22f-4b60-8051-153165a35bc3'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.004868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ff96cbd85448b48f56443d9c817956', 'request_id': 'req-a1dc7bf9-c22f-4b60-8051-153165a35bc3'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.017989 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] The token cache is being invalidate because project 32ff96cbd85448b48f56443d9c817956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:02.018569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.019229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.020841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1351/2698] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:01 2022] DELETE /identity/v3/projects/32ff96cbd85448b48f56443d9c817956 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:02.024869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7816a2-2478-4887-9ac0-133b15c9f913 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:02.034703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7816a2-2478-4887-9ac0-133b15c9f913 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:02.052796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:02.092309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:02.093196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:02.093474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:02.093748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] PATH_INFO: `/v3/users/ec16e6db4233455cb4a8a6ad5d74fda4` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:02.094965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec16e6db4233455cb4a8a6ad5d74fda4)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:02.096980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:02.147092 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec16e6db4233455cb4a8a6ad5d74fda4'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.154499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec16e6db4233455cb4a8a6ad5d74fda4'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.163763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec16e6db4233455cb4a8a6ad5d74fda4'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.164411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec16e6db4233455cb4a8a6ad5d74fda4'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.165177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-c870ed54-d808-4417-8562-a6a011f82b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec16e6db4233455cb4a8a6ad5d74fda4'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.182993 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1348/2699] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:02 2022] DELETE /identity/v3/users/ec16e6db4233455cb4a8a6ad5d74fda4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:02.471958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:02.473548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:02.488638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc7bf9-c22f-4b60-8051-153165a35bc3 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:02.552711 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:02.553928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:02.554336 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:02.554745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] PATH_INFO: `/v3/projects/e49762372ee34da6933b73000dd8389f` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:02.556370 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e49762372ee34da6933b73000dd8389f)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:02.559428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:02.616192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e49762372ee34da6933b73000dd8389f', 'request_id': 'req-0ae9283a-f701-48a5-8e09-756011517b64'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.626109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e49762372ee34da6933b73000dd8389f', 'request_id': 'req-0ae9283a-f701-48a5-8e09-756011517b64'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.633354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e49762372ee34da6933b73000dd8389f', 'request_id': 'req-0ae9283a-f701-48a5-8e09-756011517b64'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.649360 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] The token cache is being invalidate because project e49762372ee34da6933b73000dd8389f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:02.649653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.650340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0ae9283a-f701-48a5-8e09-756011517b64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:02.652036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1352/2700] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:02 2022] DELETE /identity/v3/projects/e49762372ee34da6933b73000dd8389f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:02.657233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ae382-ed25-4c02-b881-19b13620735d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:02.657704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ae382-ed25-4c02-b881-19b13620735d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:02.658233 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ae382-ed25-4c02-b881-19b13620735d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:02.677668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-465ae382-ed25-4c02-b881-19b13620735d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:02.678933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-465ae382-ed25-4c02-b881-19b13620735d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:02.742915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1349/2701] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:02.750860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f895713-ab6b-4717-ad11-f7eb4f8b8ee4 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:02.751147 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f895713-ab6b-4717-ad11-f7eb4f8b8ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:02.751975 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f895713-ab6b-4717-ad11-f7eb4f8b8ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:02.770403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-4f895713-ab6b-4717-ad11-f7eb4f8b8ee4 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:02.771953 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4f895713-ab6b-4717-ad11-f7eb4f8b8ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:02.780886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1353/2702] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:02.785300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ae382-ed25-4c02-b881-19b13620735d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:02.795689 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ae382-ed25-4c02-b881-19b13620735d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:02.800631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4961c2-752a-4423-96a3-540e7fdccf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:02.801301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4961c2-752a-4423-96a3-540e7fdccf8f admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:02.801579 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4961c2-752a-4423-96a3-540e7fdccf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:02.801846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4961c2-752a-4423-96a3-540e7fdccf8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:02.802551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4961c2-752a-4423-96a3-540e7fdccf8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:30:02.802854 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4961c2-752a-4423-96a3-540e7fdccf8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:02.804477 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4961c2-752a-4423-96a3-540e7fdccf8f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:02.816605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1350/2703] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:30:02.822663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f895713-ab6b-4717-ad11-f7eb4f8b8ee4 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:02.832810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f895713-ab6b-4717-ad11-f7eb4f8b8ee4 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:02.837958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcede00-ec20-437d-9b48-308a7b59cccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:02.838619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcede00-ec20-437d-9b48-308a7b59cccc admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:02.838900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcede00-ec20-437d-9b48-308a7b59cccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:02.839173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcede00-ec20-437d-9b48-308a7b59cccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:02.839901 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcede00-ec20-437d-9b48-308a7b59cccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:02.841817 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcede00-ec20-437d-9b48-308a7b59cccc admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:02.873177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1354/2704] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:02 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:02.877349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4961c2-752a-4423-96a3-540e7fdccf8f admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:02.888618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4961c2-752a-4423-96a3-540e7fdccf8f admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:02.925128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:02.957842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549cad0-04f0-4bf5-a585-a01f629bd2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:02.958455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549cad0-04f0-4bf5-a585-a01f629bd2fe admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:02.958767 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549cad0-04f0-4bf5-a585-a01f629bd2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:02.959048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549cad0-04f0-4bf5-a585-a01f629bd2fe admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:02.959813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549cad0-04f0-4bf5-a585-a01f629bd2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:02.961748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549cad0-04f0-4bf5-a585-a01f629bd2fe admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:02.986208 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1351/2705] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:02 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:02.988285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcede00-ec20-437d-9b48-308a7b59cccc admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:02.998742 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcede00-ec20-437d-9b48-308a7b59cccc admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:03.003973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd534ad5-1399-4a45-bdfe-f8e9cf9fddfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:03.004643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd534ad5-1399-4a45-bdfe-f8e9cf9fddfe admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.004926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd534ad5-1399-4a45-bdfe-f8e9cf9fddfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.005201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd534ad5-1399-4a45-bdfe-f8e9cf9fddfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.005931 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd534ad5-1399-4a45-bdfe-f8e9cf9fddfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:03.007613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd534ad5-1399-4a45-bdfe-f8e9cf9fddfe admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:03.025649 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1355/2706] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:03.025773 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549cad0-04f0-4bf5-a585-a01f629bd2fe admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:03.033501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549cad0-04f0-4bf5-a585-a01f629bd2fe admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:03.038704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae098e-a528-481c-a5d7-d17003d22fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:03.039387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae098e-a528-481c-a5d7-d17003d22fae admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.039675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae098e-a528-481c-a5d7-d17003d22fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.039952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae098e-a528-481c-a5d7-d17003d22fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.040693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae098e-a528-481c-a5d7-d17003d22fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:03.042305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae098e-a528-481c-a5d7-d17003d22fae admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:03.053359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1352/2707] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:03.058453 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd534ad5-1399-4a45-bdfe-f8e9cf9fddfe admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:03.071169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd534ad5-1399-4a45-bdfe-f8e9cf9fddfe admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:03.076723 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27b1f7-5d90-4f40-97aa-6c81d40a8174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:03.077608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27b1f7-5d90-4f40-97aa-6c81d40a8174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.077894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27b1f7-5d90-4f40-97aa-6c81d40a8174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.078172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27b1f7-5d90-4f40-97aa-6c81d40a8174 admin admin] PATH_INFO: `/v3/projects/6c339bf1d9494e4ba68d5ed22785c853/users/6bb8de2fc9dd4e0c96f07ff02a73dd8e/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.093501 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27b1f7-5d90-4f40-97aa-6c81d40a8174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c339bf1d9494e4ba68d5ed22785c853, user_id=6bb8de2fc9dd4e0c96f07ff02a73dd8e, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:03.096321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27b1f7-5d90-4f40-97aa-6c81d40a8174 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:03.098542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:03.118009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1356/2708] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:03 2022] PUT /identity/v3/projects/6c339bf1d9494e4ba68d5ed22785c853/users/6bb8de2fc9dd4e0c96f07ff02a73dd8e/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:03.129115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271502a-12fc-4dba-99d5-69259fa36b35 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.129398 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271502a-12fc-4dba-99d5-69259fa36b35 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.129680 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271502a-12fc-4dba-99d5-69259fa36b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.157407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-9271502a-12fc-4dba-99d5-69259fa36b35 None None] MFA Rules not processed for user `6bb8de2fc9dd4e0c96f07ff02a73dd8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:03.158620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-9271502a-12fc-4dba-99d5-69259fa36b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:03.224202 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1353/2709] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:03.228443 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27b1f7-5d90-4f40-97aa-6c81d40a8174 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:03.293872 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27b1f7-5d90-4f40-97aa-6c81d40a8174 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:03.299187 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadc645-926c-4e00-987f-da54997f86ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:03.299889 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadc645-926c-4e00-987f-da54997f86ae admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.300195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadc645-926c-4e00-987f-da54997f86ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.300473 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadc645-926c-4e00-987f-da54997f86ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.301159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadc645-926c-4e00-987f-da54997f86ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:03.303083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadc645-926c-4e00-987f-da54997f86ae admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:03.334037 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1357/2710] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:03 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:03.339448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271502a-12fc-4dba-99d5-69259fa36b35 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:03.350870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271502a-12fc-4dba-99d5-69259fa36b35 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:03.409164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a924745-4f05-4de3-97cd-533f3f737bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:03.409836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a924745-4f05-4de3-97cd-533f3f737bde admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.410113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a924745-4f05-4de3-97cd-533f3f737bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.410410 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a924745-4f05-4de3-97cd-533f3f737bde admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.411090 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a924745-4f05-4de3-97cd-533f3f737bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:03.413045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a924745-4f05-4de3-97cd-533f3f737bde admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:03.435946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1354/2711] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:03 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:03.440019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadc645-926c-4e00-987f-da54997f86ae admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:03.450865 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadc645-926c-4e00-987f-da54997f86ae admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:03.457118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdafab-2ba1-40dc-bf4e-80dbeba72f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:03.457850 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdafab-2ba1-40dc-bf4e-80dbeba72f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.458133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdafab-2ba1-40dc-bf4e-80dbeba72f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.458426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdafab-2ba1-40dc-bf4e-80dbeba72f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.459161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdafab-2ba1-40dc-bf4e-80dbeba72f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:03.461097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdafab-2ba1-40dc-bf4e-80dbeba72f58 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:03.474718 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1358/2712] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:03.478810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a924745-4f05-4de3-97cd-533f3f737bde admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:03.489564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a924745-4f05-4de3-97cd-533f3f737bde admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:03.495563 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be71523-95b7-4996-9056-174555fcd356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:03.496447 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be71523-95b7-4996-9056-174555fcd356 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.496754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be71523-95b7-4996-9056-174555fcd356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.497025 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be71523-95b7-4996-9056-174555fcd356 admin admin] PATH_INFO: `/v3/projects/587879a4f8d74b159f8c7288276ca607/users/561e7988504d47df81cc89efdb0ee932/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.517401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be71523-95b7-4996-9056-174555fcd356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587879a4f8d74b159f8c7288276ca607, user_id=561e7988504d47df81cc89efdb0ee932, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:03.521236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be71523-95b7-4996-9056-174555fcd356 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:03.540334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1355/2713] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:03 2022] PUT /identity/v3/projects/587879a4f8d74b159f8c7288276ca607/users/561e7988504d47df81cc89efdb0ee932/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:03.544549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdafab-2ba1-40dc-bf4e-80dbeba72f58 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:03.556298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdafab-2ba1-40dc-bf4e-80dbeba72f58 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:03.621544 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6cc0f-ec11-4992-ab48-3ce55ce54b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:03.622228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6cc0f-ec11-4992-ab48-3ce55ce54b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.622514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6cc0f-ec11-4992-ab48-3ce55ce54b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.622792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6cc0f-ec11-4992-ab48-3ce55ce54b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.623574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6cc0f-ec11-4992-ab48-3ce55ce54b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:03.625196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6cc0f-ec11-4992-ab48-3ce55ce54b6f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:03.638638 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1359/2714] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:03.642721 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be71523-95b7-4996-9056-174555fcd356 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:03.648554 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:03.655514 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be71523-95b7-4996-9056-174555fcd356 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:03.660425 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3447f59-ad73-4939-8cde-4388123b099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:03.661336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3447f59-ad73-4939-8cde-4388123b099c admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.661621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3447f59-ad73-4939-8cde-4388123b099c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.661899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3447f59-ad73-4939-8cde-4388123b099c admin admin] PATH_INFO: `/v3/projects/587879a4f8d74b159f8c7288276ca607/users/561e7988504d47df81cc89efdb0ee932/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.663912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3447f59-ad73-4939-8cde-4388123b099c admin admin] RBAC: Authorizing `identity:create_grant(project_id=587879a4f8d74b159f8c7288276ca607, user_id=561e7988504d47df81cc89efdb0ee932, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:03.666622 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3447f59-ad73-4939-8cde-4388123b099c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:03.678733 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1356/2715] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:03 2022] PUT /identity/v3/projects/587879a4f8d74b159f8c7288276ca607/users/561e7988504d47df81cc89efdb0ee932/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:03.690248 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8c55b-5f53-4d13-be8e-be934dac07c1 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.690540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8c55b-5f53-4d13-be8e-be934dac07c1 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.690816 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8c55b-5f53-4d13-be8e-be934dac07c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.719068 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-96d8c55b-5f53-4d13-be8e-be934dac07c1 None None] MFA Rules not processed for user `561e7988504d47df81cc89efdb0ee932`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:03.720364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-96d8c55b-5f53-4d13-be8e-be934dac07c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:03.773896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:03.794713 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1360/2716] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:03.807406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3447f59-ad73-4939-8cde-4388123b099c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:03.809539 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c3447f59-ad73-4939-8cde-4388123b099c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:03.879612 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3447f59-ad73-4939-8cde-4388123b099c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:03.883551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922ebf5-b9ad-4956-bcc4-8ce8c7eaafdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:03.884269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922ebf5-b9ad-4956-bcc4-8ce8c7eaafdc service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:03.884556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922ebf5-b9ad-4956-bcc4-8ce8c7eaafdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:03.884838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922ebf5-b9ad-4956-bcc4-8ce8c7eaafdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:03.899156 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922ebf5-b9ad-4956-bcc4-8ce8c7eaafdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:03.902438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922ebf5-b9ad-4956-bcc4-8ce8c7eaafdc service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:03.917475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1357/2717] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:04.020150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8c55b-5f53-4d13-be8e-be934dac07c1 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:04.032563 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8c55b-5f53-4d13-be8e-be934dac07c1 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:04.037064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe42f0-9a1b-4719-914a-1200e1842878 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:04.037856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe42f0-9a1b-4719-914a-1200e1842878 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:04.039631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe42f0-9a1b-4719-914a-1200e1842878 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:04.040033 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe42f0-9a1b-4719-914a-1200e1842878 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:04.103754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922ebf5-b9ad-4956-bcc4-8ce8c7eaafdc service cinder] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:04.105587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4922ebf5-b9ad-4956-bcc4-8ce8c7eaafdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:04.145586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe42f0-9a1b-4719-914a-1200e1842878 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:04.147305 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe42f0-9a1b-4719-914a-1200e1842878 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:04.159340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1361/2718] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:04.187292 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922ebf5-b9ad-4956-bcc4-8ce8c7eaafdc service cinder] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:04.192056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:04.192962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:04.193915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:04.193915 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] PATH_INFO: `/v3/users/c2fa14b07aa842f3b00cf1809ce676de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:04.194920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2fa14b07aa842f3b00cf1809ce676de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:04.196953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:04.256131 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2fa14b07aa842f3b00cf1809ce676de'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:04.267552 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2fa14b07aa842f3b00cf1809ce676de'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:04.276754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2fa14b07aa842f3b00cf1809ce676de'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:04.277465 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2fa14b07aa842f3b00cf1809ce676de'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:04.278177 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2fa14b07aa842f3b00cf1809ce676de'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:04.300900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1358/2719] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:04 2022] DELETE /identity/v3/users/c2fa14b07aa842f3b00cf1809ce676de => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:04.305434 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1a45-aa4b-4f1c-b4cf-b04d42e9d360 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:04.305729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1a45-aa4b-4f1c-b4cf-b04d42e9d360 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:04.306113 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1a45-aa4b-4f1c-b4cf-b04d42e9d360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:04.325503 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-5a8d1a45-aa4b-4f1c-b4cf-b04d42e9d360 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:04.326957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5a8d1a45-aa4b-4f1c-b4cf-b04d42e9d360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:04.391338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1362/2720] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:04.401389 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:04.402830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:04.418696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:04.477972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c93cde-e5e1-4844-960d-e7d7553b27a7 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:04.481620 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d123e-c01e-44e1-9ffc-87070a22c373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:04.482282 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d123e-c01e-44e1-9ffc-87070a22c373 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:04.482587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d123e-c01e-44e1-9ffc-87070a22c373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:04.482821 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d123e-c01e-44e1-9ffc-87070a22c373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:04.492142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d123e-c01e-44e1-9ffc-87070a22c373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:04.493584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d123e-c01e-44e1-9ffc-87070a22c373 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:04.503870 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1359/2721] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:04.782122 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1a45-aa4b-4f1c-b4cf-b04d42e9d360 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:04.783779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-5a8d1a45-aa4b-4f1c-b4cf-b04d42e9d360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:04.797117 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1a45-aa4b-4f1c-b4cf-b04d42e9d360 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:04.802095 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:04.802977 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:04.803272 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:04.803572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] PATH_INFO: `/v3/projects/5eaf4b3186ab472da0cd67bf31fd5b03` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:04.804769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eaf4b3186ab472da0cd67bf31fd5b03)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:04.806857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:04.869164 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eaf4b3186ab472da0cd67bf31fd5b03', 'request_id': 'req-0af21ac6-33e6-4922-b7d5-25393d45e1ef'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:04.875783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eaf4b3186ab472da0cd67bf31fd5b03', 'request_id': 'req-0af21ac6-33e6-4922-b7d5-25393d45e1ef'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:04.882524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eaf4b3186ab472da0cd67bf31fd5b03', 'request_id': 'req-0af21ac6-33e6-4922-b7d5-25393d45e1ef'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:04.895268 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] The token cache is being invalidate because project 5eaf4b3186ab472da0cd67bf31fd5b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:04.895582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:04.896192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:04.897820 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1363/2722] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:04 2022] DELETE /identity/v3/projects/5eaf4b3186ab472da0cd67bf31fd5b03 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:04.902040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d123e-c01e-44e1-9ffc-87070a22c373 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:04.913314 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d123e-c01e-44e1-9ffc-87070a22c373 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:04.981880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:04.982760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:04.983044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:04.983343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] PATH_INFO: `/v3/users/7a216d37e89f447f83ffc7dffb068dea` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:04.984589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a216d37e89f447f83ffc7dffb068dea)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:04.986991 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:05.049568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a216d37e89f447f83ffc7dffb068dea'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.059572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a216d37e89f447f83ffc7dffb068dea'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.069482 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a216d37e89f447f83ffc7dffb068dea'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.070194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a216d37e89f447f83ffc7dffb068dea'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.070956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a216d37e89f447f83ffc7dffb068dea'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.090420 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1360/2723] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:04 2022] DELETE /identity/v3/users/7a216d37e89f447f83ffc7dffb068dea => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:05.404307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:05.405900 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:05.428590 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af21ac6-33e6-4922-b7d5-25393d45e1ef admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:05.439347 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:05.491784 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:05.492664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:05.492935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:05.493200 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] PATH_INFO: `/v3/projects/c1f03404ac12487088812ab1a9941a68` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:05.494400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f03404ac12487088812ab1a9941a68)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:05.496365 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:05.551280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f03404ac12487088812ab1a9941a68', 'request_id': 'req-35093e56-6725-4743-885d-bc960b30bfa6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.557646 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f03404ac12487088812ab1a9941a68', 'request_id': 'req-35093e56-6725-4743-885d-bc960b30bfa6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.564318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f03404ac12487088812ab1a9941a68', 'request_id': 'req-35093e56-6725-4743-885d-bc960b30bfa6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.579926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] The token cache is being invalidate because project c1f03404ac12487088812ab1a9941a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:05.580237 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.580986 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.582930 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1364/2724] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:05 2022] DELETE /identity/v3/projects/c1f03404ac12487088812ab1a9941a68 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:05.586778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:05.588742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:05.601899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1270f08-b644-49cf-b820-10e9a0022066 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:05.633926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:05.663908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:05.664807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:05.665089 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:05.665364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] PATH_INFO: `/v3/users/3a50cd44d7ff4b05803e8c7d2bf779a8` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:05.666547 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a50cd44d7ff4b05803e8c7d2bf779a8)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:05.668534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:05.722386 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a50cd44d7ff4b05803e8c7d2bf779a8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.734785 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a50cd44d7ff4b05803e8c7d2bf779a8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.745891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a50cd44d7ff4b05803e8c7d2bf779a8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.746642 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a50cd44d7ff4b05803e8c7d2bf779a8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.747446 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e3907f92-4033-4c4a-8949-b36542c6c6ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a50cd44d7ff4b05803e8c7d2bf779a8'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:05.766550 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1361/2725] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:05 2022] DELETE /identity/v3/users/3a50cd44d7ff4b05803e8c7d2bf779a8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:06.041547 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:06.044010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:06.058635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35093e56-6725-4743-885d-bc960b30bfa6 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:06.131759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:06.132644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.132923 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.133181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] PATH_INFO: `/v3/projects/ab4f344d551d41fa8691d70c229c65ce` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.134393 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab4f344d551d41fa8691d70c229c65ce)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:06.136338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:06.192313 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:06.196788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab4f344d551d41fa8691d70c229c65ce', 'request_id': 'req-b796c245-b386-43ca-8173-0a5d28143a9e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:06.207022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab4f344d551d41fa8691d70c229c65ce', 'request_id': 'req-b796c245-b386-43ca-8173-0a5d28143a9e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:06.215944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab4f344d551d41fa8691d70c229c65ce', 'request_id': 'req-b796c245-b386-43ca-8173-0a5d28143a9e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:06.232549 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] The token cache is being invalidate because project ab4f344d551d41fa8691d70c229c65ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:06.232911 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:06.233756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b796c245-b386-43ca-8173-0a5d28143a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:06.237262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1365/2726] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:06 2022] DELETE /identity/v3/projects/ab4f344d551d41fa8691d70c229c65ce => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:06.242143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f34d4-666a-4d7f-8df6-fd7eaf17951b None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.242629 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f34d4-666a-4d7f-8df6-fd7eaf17951b None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.242723 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f34d4-666a-4d7f-8df6-fd7eaf17951b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.270587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-6c1f34d4-666a-4d7f-8df6-fd7eaf17951b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:06.271861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-6c1f34d4-666a-4d7f-8df6-fd7eaf17951b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:06.330152 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1362/2727] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:06.337890 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaac6b-8bed-439b-a6f1-bb3828fac6a0 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.338180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaac6b-8bed-439b-a6f1-bb3828fac6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.338456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaac6b-8bed-439b-a6f1-bb3828fac6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.357214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-05aaac6b-8bed-439b-a6f1-bb3828fac6a0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:06.358493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-05aaac6b-8bed-439b-a6f1-bb3828fac6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:06.367605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1366/2728] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:06.371987 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f34d4-666a-4d7f-8df6-fd7eaf17951b None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:06.382140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f34d4-666a-4d7f-8df6-fd7eaf17951b None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:06.387194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7cb93-29b4-406e-9304-bb88e758cd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:06.387904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7cb93-29b4-406e-9304-bb88e758cd0a admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.388188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7cb93-29b4-406e-9304-bb88e758cd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.388483 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7cb93-29b4-406e-9304-bb88e758cd0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.389226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7cb93-29b4-406e-9304-bb88e758cd0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:30:06.389536 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7cb93-29b4-406e-9304-bb88e758cd0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:06.391140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7cb93-29b4-406e-9304-bb88e758cd0a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:06.403429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1363/2729] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:30:06.408424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aaac6b-8bed-439b-a6f1-bb3828fac6a0 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:06.418297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aaac6b-8bed-439b-a6f1-bb3828fac6a0 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:06.424009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe562f-c0e4-4a79-b6a4-835a8a486402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:06.424684 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe562f-c0e4-4a79-b6a4-835a8a486402 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.424962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe562f-c0e4-4a79-b6a4-835a8a486402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.425231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe562f-c0e4-4a79-b6a4-835a8a486402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.425945 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe562f-c0e4-4a79-b6a4-835a8a486402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:06.427925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe562f-c0e4-4a79-b6a4-835a8a486402 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:06.462858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1367/2730] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:06 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:30:06.464154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7cb93-29b4-406e-9304-bb88e758cd0a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:06.474305 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7cb93-29b4-406e-9304-bb88e758cd0a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:06.530011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:06.550500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5677af2-dd94-4834-9a89-cc1bd703f660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:06.551185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5677af2-dd94-4834-9a89-cc1bd703f660 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.551492 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5677af2-dd94-4834-9a89-cc1bd703f660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.551806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5677af2-dd94-4834-9a89-cc1bd703f660 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.552453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5677af2-dd94-4834-9a89-cc1bd703f660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:06.554370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5677af2-dd94-4834-9a89-cc1bd703f660 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:06.580481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1364/2731] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:06 2022] POST /identity/v3/users => generated 284 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:06.584829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe562f-c0e4-4a79-b6a4-835a8a486402 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:06.596163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe562f-c0e4-4a79-b6a4-835a8a486402 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:06.601561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16d2d-039e-4f7c-b119-b0ce4817a7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:06.602238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16d2d-039e-4f7c-b119-b0ce4817a7dc admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.602519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16d2d-039e-4f7c-b119-b0ce4817a7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.602797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16d2d-039e-4f7c-b119-b0ce4817a7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.604023 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c16d2d-039e-4f7c-b119-b0ce4817a7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:06.606327 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c16d2d-039e-4f7c-b119-b0ce4817a7dc admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:06.619203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1368/2732] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:06.625536 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5677af2-dd94-4834-9a89-cc1bd703f660 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:06.640575 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5677af2-dd94-4834-9a89-cc1bd703f660 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:06.646649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddf2b6-1813-4271-b8e6-32fc3f8d6f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:06.647411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf2b6-1813-4271-b8e6-32fc3f8d6f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.647713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf2b6-1813-4271-b8e6-32fc3f8d6f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.647980 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf2b6-1813-4271-b8e6-32fc3f8d6f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.648751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddf2b6-1813-4271-b8e6-32fc3f8d6f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:06.650404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddf2b6-1813-4271-b8e6-32fc3f8d6f1e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:06.660976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1365/2733] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:06.665281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16d2d-039e-4f7c-b119-b0ce4817a7dc admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:06.677130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16d2d-039e-4f7c-b119-b0ce4817a7dc admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:06.682551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2238aa-b2f2-419d-99db-f695a2d0fa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:06.683339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2238aa-b2f2-419d-99db-f695a2d0fa30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.683627 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2238aa-b2f2-419d-99db-f695a2d0fa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.683902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2238aa-b2f2-419d-99db-f695a2d0fa30 admin admin] PATH_INFO: `/v3/projects/e16ce9238fc64bfe914286f4e429fa4c/users/1a1e8da20a9144fc8de9e40e77ef1f51/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.700005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2238aa-b2f2-419d-99db-f695a2d0fa30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e16ce9238fc64bfe914286f4e429fa4c, user_id=1a1e8da20a9144fc8de9e40e77ef1f51, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:06.702750 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2238aa-b2f2-419d-99db-f695a2d0fa30 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:06.719440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1369/2734] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:06 2022] PUT /identity/v3/projects/e16ce9238fc64bfe914286f4e429fa4c/users/1a1e8da20a9144fc8de9e40e77ef1f51/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:06.731134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7376a0-f1fb-407c-bc8d-7e51ace01d39 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.731462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7376a0-f1fb-407c-bc8d-7e51ace01d39 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.731742 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7376a0-f1fb-407c-bc8d-7e51ace01d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.760537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-3e7376a0-f1fb-407c-bc8d-7e51ace01d39 None None] MFA Rules not processed for user `1a1e8da20a9144fc8de9e40e77ef1f51`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:06.761800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3e7376a0-f1fb-407c-bc8d-7e51ace01d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:06.832290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1366/2735] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:06.844005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2238aa-b2f2-419d-99db-f695a2d0fa30 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:06.846034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1d2238aa-b2f2-419d-99db-f695a2d0fa30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:06.935022 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2238aa-b2f2-419d-99db-f695a2d0fa30 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:06.938888 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7237d3a-3d44-490c-b1a9-cf59ce156cfa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab33473d40094ec681e538cde250ec2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:06.939622 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237d3a-3d44-490c-b1a9-cf59ce156cfa service glance] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:06.939996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237d3a-3d44-490c-b1a9-cf59ce156cfa service glance] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:06.940143 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237d3a-3d44-490c-b1a9-cf59ce156cfa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:06.950398 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7237d3a-3d44-490c-b1a9-cf59ce156cfa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:06.952356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7237d3a-3d44-490c-b1a9-cf59ce156cfa service glance] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:06.963017 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1370/2736] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:07.354627 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7376a0-f1fb-407c-bc8d-7e51ace01d39 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:07.356142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3e7376a0-f1fb-407c-bc8d-7e51ace01d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:07.451936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7376a0-f1fb-407c-bc8d-7e51ace01d39 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:07.460424 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:07.461346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:07.461630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:07.462072 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] PATH_INFO: `/v3/users/444d988d8982463bb37a9a51950705d5` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:07.463811 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=444d988d8982463bb37a9a51950705d5)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:07.466684 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:07.540276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '444d988d8982463bb37a9a51950705d5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:07.542505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:07.549098 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '444d988d8982463bb37a9a51950705d5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:07.562219 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '444d988d8982463bb37a9a51950705d5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:07.562990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '444d988d8982463bb37a9a51950705d5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:07.563801 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '444d988d8982463bb37a9a51950705d5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:07.590380 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1367/2737] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:07 2022] DELETE /identity/v3/users/444d988d8982463bb37a9a51950705d5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:07.597971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf8633-1f4a-49f9-95d2-2a33b3c4b01b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:07.598438 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf8633-1f4a-49f9-95d2-2a33b3c4b01b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:07.598438 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf8633-1f4a-49f9-95d2-2a33b3c4b01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:07.628426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-41bf8633-1f4a-49f9-95d2-2a33b3c4b01b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:07.630363 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-41bf8633-1f4a-49f9-95d2-2a33b3c4b01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:07.700405 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:07.718525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1371/2738] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:07.731740 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:07.733175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:07.801041 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17184058-acb4-43b6-8d3c-2d14d5a28395 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:07.805066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd35bb5-6942-4eff-bb7b-1da1384461dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:07.805731 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd35bb5-6942-4eff-bb7b-1da1384461dd service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:07.806011 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd35bb5-6942-4eff-bb7b-1da1384461dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:07.806293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd35bb5-6942-4eff-bb7b-1da1384461dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:07.816972 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd35bb5-6942-4eff-bb7b-1da1384461dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:07.818810 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd35bb5-6942-4eff-bb7b-1da1384461dd service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:07.860169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1368/2739] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:08.169538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf8633-1f4a-49f9-95d2-2a33b3c4b01b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:08.171559 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-41bf8633-1f4a-49f9-95d2-2a33b3c4b01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:08.191878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf8633-1f4a-49f9-95d2-2a33b3c4b01b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:08.198893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:08.200488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:08.200780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:08.201062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] PATH_INFO: `/v3/projects/574dca6a9bf744698378feacb0c10c86` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:08.202345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=574dca6a9bf744698378feacb0c10c86)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:08.204584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:08.262277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574dca6a9bf744698378feacb0c10c86', 'request_id': 'req-bb84a127-398d-4a4f-886a-e083227eb6a4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:08.270162 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574dca6a9bf744698378feacb0c10c86', 'request_id': 'req-bb84a127-398d-4a4f-886a-e083227eb6a4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:08.276128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574dca6a9bf744698378feacb0c10c86', 'request_id': 'req-bb84a127-398d-4a4f-886a-e083227eb6a4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:08.290483 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] The token cache is being invalidate because project 574dca6a9bf744698378feacb0c10c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:08.290769 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:08.291409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb84a127-398d-4a4f-886a-e083227eb6a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:08.292976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1372/2740] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:08 2022] DELETE /identity/v3/projects/574dca6a9bf744698378feacb0c10c86 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:08.824175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd35bb5-6942-4eff-bb7b-1da1384461dd service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:08.825655 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-acd35bb5-6942-4eff-bb7b-1da1384461dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:08.840771 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd35bb5-6942-4eff-bb7b-1da1384461dd service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:08.906430 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:08.907362 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:08.907647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:08.907947 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] PATH_INFO: `/v3/users/d320afdb49a64af1b5dbdfdb52e37d90` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:08.909174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d320afdb49a64af1b5dbdfdb52e37d90)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:08.911133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:08.965024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd320afdb49a64af1b5dbdfdb52e37d90'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:08.972337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd320afdb49a64af1b5dbdfdb52e37d90'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:08.981325 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd320afdb49a64af1b5dbdfdb52e37d90'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:08.982164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd320afdb49a64af1b5dbdfdb52e37d90'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:08.982807 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd320afdb49a64af1b5dbdfdb52e37d90'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:08.990687 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:09.004379 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1369/2741] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:08 2022] DELETE /identity/v3/users/d320afdb49a64af1b5dbdfdb52e37d90 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:09.009120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d137457-1fc5-4f99-a370-32aaa9eaafb2 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.009601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d137457-1fc5-4f99-a370-32aaa9eaafb2 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.009601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d137457-1fc5-4f99-a370-32aaa9eaafb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.029630 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6d137457-1fc5-4f99-a370-32aaa9eaafb2 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:09.031045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6d137457-1fc5-4f99-a370-32aaa9eaafb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:09.096091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1373/2742] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:09.107634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:09.114241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:09.150386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d137457-1fc5-4f99-a370-32aaa9eaafb2 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:09.152108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6d137457-1fc5-4f99-a370-32aaa9eaafb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:09.166226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d137457-1fc5-4f99-a370-32aaa9eaafb2 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:09.176635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:09.177685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.177968 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.178299 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] PATH_INFO: `/v3/users/6bb8de2fc9dd4e0c96f07ff02a73dd8e` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.179564 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9630c73-de62-4a56-8963-6e1f31bae009 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:09.179780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb8de2fc9dd4e0c96f07ff02a73dd8e)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:09.181883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:09.183703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915091c-bdb7-4c7a-8dcd-5269fce1f136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:09.184144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915091c-bdb7-4c7a-8dcd-5269fce1f136 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.184427 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915091c-bdb7-4c7a-8dcd-5269fce1f136 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.184701 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915091c-bdb7-4c7a-8dcd-5269fce1f136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.195383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915091c-bdb7-4c7a-8dcd-5269fce1f136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:09.196787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915091c-bdb7-4c7a-8dcd-5269fce1f136 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:09.212188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1370/2743] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:09.237325 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb8de2fc9dd4e0c96f07ff02a73dd8e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.244303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb8de2fc9dd4e0c96f07ff02a73dd8e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.245003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb8de2fc9dd4e0c96f07ff02a73dd8e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.254564 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb8de2fc9dd4e0c96f07ff02a73dd8e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.260396 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:09.269156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb8de2fc9dd4e0c96f07ff02a73dd8e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.276447 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1374/2744] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:09 2022] DELETE /identity/v3/users/6bb8de2fc9dd4e0c96f07ff02a73dd8e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:09.281005 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddca2f3-0dfa-4737-8199-103ec9e87bc6 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.281291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddca2f3-0dfa-4737-8199-103ec9e87bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.281587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddca2f3-0dfa-4737-8199-103ec9e87bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.299828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-4ddca2f3-0dfa-4737-8199-103ec9e87bc6 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:09.301161 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4ddca2f3-0dfa-4737-8199-103ec9e87bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:09.365873 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1371/2745] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:09.375201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:09.376745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:09.439171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e27563-9c3a-478a-82d2-94791e681cf7 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:09.442922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05452640-2e8a-446f-a29d-6a2238fb10c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:09.443604 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05452640-2e8a-446f-a29d-6a2238fb10c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.443885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05452640-2e8a-446f-a29d-6a2238fb10c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.444156 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05452640-2e8a-446f-a29d-6a2238fb10c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.453444 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05452640-2e8a-446f-a29d-6a2238fb10c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:09.455071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05452640-2e8a-446f-a29d-6a2238fb10c8 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:09.464662 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1375/2746] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:09.498181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddca2f3-0dfa-4737-8199-103ec9e87bc6 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:09.503372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4ddca2f3-0dfa-4737-8199-103ec9e87bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:09.517812 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddca2f3-0dfa-4737-8199-103ec9e87bc6 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:09.533171 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:09.534313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.534806 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.535102 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] PATH_INFO: `/v3/projects/3d7bdc049109419fb0b7188730aeef9f` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.536429 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d7bdc049109419fb0b7188730aeef9f)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:09.538568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:09.602711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d7bdc049109419fb0b7188730aeef9f', 'request_id': 'req-7271bd1c-0294-4adf-9b9f-9282c147c73d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.609607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d7bdc049109419fb0b7188730aeef9f', 'request_id': 'req-7271bd1c-0294-4adf-9b9f-9282c147c73d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.619899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d7bdc049109419fb0b7188730aeef9f', 'request_id': 'req-7271bd1c-0294-4adf-9b9f-9282c147c73d'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.636862 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] The token cache is being invalidate because project 3d7bdc049109419fb0b7188730aeef9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:09.637157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.643720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7271bd1c-0294-4adf-9b9f-9282c147c73d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.645748 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1372/2747] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:09 2022] DELETE /identity/v3/projects/3d7bdc049109419fb0b7188730aeef9f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:09.650636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f54bd-cff1-4e10-9c23-3726e275b043 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.650929 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f54bd-cff1-4e10-9c23-3726e275b043 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.651199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f54bd-cff1-4e10-9c23-3726e275b043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.669135 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-1d4f54bd-cff1-4e10-9c23-3726e275b043 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:09.670359 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1d4f54bd-cff1-4e10-9c23-3726e275b043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:09.737699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1376/2748] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:09.745577 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b01469-2f52-4065-be2e-979d06a9bd25 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.745946 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b01469-2f52-4065-be2e-979d06a9bd25 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.746295 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b01469-2f52-4065-be2e-979d06a9bd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.766403 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-55b01469-2f52-4065-be2e-979d06a9bd25 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:09.767842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-55b01469-2f52-4065-be2e-979d06a9bd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:09.777455 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f54bd-cff1-4e10-9c23-3726e275b043 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:09.779465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1d4f54bd-cff1-4e10-9c23-3726e275b043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:09.782832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1373/2749] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:09.787615 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b01469-2f52-4065-be2e-979d06a9bd25 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:09.796494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f54bd-cff1-4e10-9c23-3726e275b043 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:09.798239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b01469-2f52-4065-be2e-979d06a9bd25 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:09.801973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:09.802979 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.803284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.803581 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] PATH_INFO: `/v3/projects/6c339bf1d9494e4ba68d5ed22785c853` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.804955 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c339bf1d9494e4ba68d5ed22785c853)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:09.805044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74d7a1-4905-4fcf-a57e-fbb772eeafed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:09.805711 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d7a1-4905-4fcf-a57e-fbb772eeafed admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.806190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d7a1-4905-4fcf-a57e-fbb772eeafed admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.806352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d7a1-4905-4fcf-a57e-fbb772eeafed admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.807075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:09.807185 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74d7a1-4905-4fcf-a57e-fbb772eeafed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:30:09.807474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74d7a1-4905-4fcf-a57e-fbb772eeafed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:09.809019 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74d7a1-4905-4fcf-a57e-fbb772eeafed admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:09.811674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:09.827288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1374/2750] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 06:30:09.828935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74d7a1-4905-4fcf-a57e-fbb772eeafed admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:09.842186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74d7a1-4905-4fcf-a57e-fbb772eeafed admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:09.846883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd3c33-fa1e-4dee-8e67-42b913a175e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:09.847599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd3c33-fa1e-4dee-8e67-42b913a175e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.847877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd3c33-fa1e-4dee-8e67-42b913a175e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.848154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd3c33-fa1e-4dee-8e67-42b913a175e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.848853 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd3c33-fa1e-4dee-8e67-42b913a175e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:09.850752 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd3c33-fa1e-4dee-8e67-42b913a175e0 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:09.860345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c339bf1d9494e4ba68d5ed22785c853', 'request_id': 'req-b402e506-d8e0-4950-9143-24c9697dc533'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.869728 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c339bf1d9494e4ba68d5ed22785c853', 'request_id': 'req-b402e506-d8e0-4950-9143-24c9697dc533'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.877863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c339bf1d9494e4ba68d5ed22785c853', 'request_id': 'req-b402e506-d8e0-4950-9143-24c9697dc533'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.887024 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1375/2751] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:09 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:09.888258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd3c33-fa1e-4dee-8e67-42b913a175e0 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:09.893194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] The token cache is being invalidate because project 6c339bf1d9494e4ba68d5ed22785c853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:09.893481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.894168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:09.895795 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1377/2752] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:09 2022] DELETE /identity/v3/projects/6c339bf1d9494e4ba68d5ed22785c853 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:09.899588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:09.900935 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:09.914768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:09.953618 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd3c33-fa1e-4dee-8e67-42b913a175e0 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:09.960022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2289bdd-90bd-4966-af6b-284068310a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:09.960717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2289bdd-90bd-4966-af6b-284068310a54 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.961001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2289bdd-90bd-4966-af6b-284068310a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.961278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2289bdd-90bd-4966-af6b-284068310a54 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.961978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2289bdd-90bd-4966-af6b-284068310a54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:09.963958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2289bdd-90bd-4966-af6b-284068310a54 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:09.972491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402e506-d8e0-4950-9143-24c9697dc533 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:09.977917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:09.979079 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:09.979389 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:09.979663 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] PATH_INFO: `/v3/users/561e7988504d47df81cc89efdb0ee932` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:09.981015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=561e7988504d47df81cc89efdb0ee932)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:09.983181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:09.986201 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1376/2753] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:09 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:09.990713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2289bdd-90bd-4966-af6b-284068310a54 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:09.993238 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b2289bdd-90bd-4966-af6b-284068310a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:10.007169 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2289bdd-90bd-4966-af6b-284068310a54 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.012607 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a9735-b81a-4b3d-8157-dee717151b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.013293 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a9735-b81a-4b3d-8157-dee717151b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.013576 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a9735-b81a-4b3d-8157-dee717151b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.013886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a9735-b81a-4b3d-8157-dee717151b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.014594 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a9735-b81a-4b3d-8157-dee717151b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.016354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a9735-b81a-4b3d-8157-dee717151b09 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.027910 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1377/2754] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:30:10.031678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a9735-b81a-4b3d-8157-dee717151b09 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.043951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a9735-b81a-4b3d-8157-dee717151b09 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.044201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '561e7988504d47df81cc89efdb0ee932'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:10.050338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd0d2-ed85-4bee-b7eb-052e7242131e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.051215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd0d2-ed85-4bee-b7eb-052e7242131e admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.051702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd0d2-ed85-4bee-b7eb-052e7242131e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.052034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '561e7988504d47df81cc89efdb0ee932'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:10.052300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd0d2-ed85-4bee-b7eb-052e7242131e admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.052748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '561e7988504d47df81cc89efdb0ee932'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:10.053383 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cd0d2-ed85-4bee-b7eb-052e7242131e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.055368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cd0d2-ed85-4bee-b7eb-052e7242131e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.067753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1378/2755] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:30:10.068093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '561e7988504d47df81cc89efdb0ee932'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:10.072000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd0d2-ed85-4bee-b7eb-052e7242131e admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.083164 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd0d2-ed85-4bee-b7eb-052e7242131e admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.088123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-02e3761b-0182-4c1c-adf6-14a3a17a14c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '561e7988504d47df81cc89efdb0ee932'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:10.088391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d7e09-fc48-4992-b9d2-1c4a3bcbd610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.089939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d7e09-fc48-4992-b9d2-1c4a3bcbd610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.090671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d7e09-fc48-4992-b9d2-1c4a3bcbd610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.090808 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d7e09-fc48-4992-b9d2-1c4a3bcbd610 admin admin] PATH_INFO: `/v3/projects/4167e0d6bb6648f5a161edd2bb0c98fa/users/a3c78a52296e4ccbbe77eb433aa558a6/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.098726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1378/2756] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:09 2022] DELETE /identity/v3/users/561e7988504d47df81cc89efdb0ee932 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:10.108994 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d7e09-fc48-4992-b9d2-1c4a3bcbd610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4167e0d6bb6648f5a161edd2bb0c98fa, user_id=a3c78a52296e4ccbbe77eb433aa558a6, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.111772 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d7e09-fc48-4992-b9d2-1c4a3bcbd610 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.137737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1379/2757] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:10 2022] PUT /identity/v3/projects/4167e0d6bb6648f5a161edd2bb0c98fa/users/a3c78a52296e4ccbbe77eb433aa558a6/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:10.148171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2f5d8-ee3c-445b-a3ba-b728316cc389 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.148354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2f5d8-ee3c-445b-a3ba-b728316cc389 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.148782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2f5d8-ee3c-445b-a3ba-b728316cc389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.178239 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-9ca2f5d8-ee3c-445b-a3ba-b728316cc389 None None] MFA Rules not processed for user `a3c78a52296e4ccbbe77eb433aa558a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:10.179515 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9ca2f5d8-ee3c-445b-a3ba-b728316cc389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:10.253658 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1379/2758] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:10.258235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d7e09-fc48-4992-b9d2-1c4a3bcbd610 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.260022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-fa5d7e09-fc48-4992-b9d2-1c4a3bcbd610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:10.334168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d7e09-fc48-4992-b9d2-1c4a3bcbd610 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.338729 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5a924-a847-4cec-ab00-c612ded27cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.339439 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5a924-a847-4cec-ab00-c612ded27cad admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.339720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5a924-a847-4cec-ab00-c612ded27cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.339990 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5a924-a847-4cec-ab00-c612ded27cad admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.340696 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5a924-a847-4cec-ab00-c612ded27cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.342613 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5a924-a847-4cec-ab00-c612ded27cad admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.371830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1380/2759] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:10 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:10.373010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2f5d8-ee3c-445b-a3ba-b728316cc389 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.384418 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2f5d8-ee3c-445b-a3ba-b728316cc389 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.403800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:10.453212 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bc2aa-5cf9-43fa-89e8-c5331a11e03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.453913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bc2aa-5cf9-43fa-89e8-c5331a11e03c admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.454255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bc2aa-5cf9-43fa-89e8-c5331a11e03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.454460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bc2aa-5cf9-43fa-89e8-c5331a11e03c admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.455294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bc2aa-5cf9-43fa-89e8-c5331a11e03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.457255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bc2aa-5cf9-43fa-89e8-c5331a11e03c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.479722 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1380/2760] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:10 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:30:10.483970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5a924-a847-4cec-ab00-c612ded27cad admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.489574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:10.495883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5a924-a847-4cec-ab00-c612ded27cad admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.501135 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fd891-46d9-490e-bc7b-ac705afdc68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.501939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fd891-46d9-490e-bc7b-ac705afdc68b admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.502606 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fd891-46d9-490e-bc7b-ac705afdc68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.502952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fd891-46d9-490e-bc7b-ac705afdc68b admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.503861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fd891-46d9-490e-bc7b-ac705afdc68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.505551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fd891-46d9-490e-bc7b-ac705afdc68b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.518154 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1381/2761] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:10.525065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bc2aa-5cf9-43fa-89e8-c5331a11e03c admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.537879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bc2aa-5cf9-43fa-89e8-c5331a11e03c admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.543570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af5545-a340-4153-8134-e8dc4bce8b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.544374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af5545-a340-4153-8134-e8dc4bce8b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.544715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af5545-a340-4153-8134-e8dc4bce8b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.545088 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af5545-a340-4153-8134-e8dc4bce8b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.545893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af5545-a340-4153-8134-e8dc4bce8b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.547648 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af5545-a340-4153-8134-e8dc4bce8b47 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.558954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1381/2762] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:10.563556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fd891-46d9-490e-bc7b-ac705afdc68b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.574730 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fd891-46d9-490e-bc7b-ac705afdc68b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.579670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80b960-ee77-498d-b5fa-677dcc0bd3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.580670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80b960-ee77-498d-b5fa-677dcc0bd3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.580952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80b960-ee77-498d-b5fa-677dcc0bd3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.581225 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80b960-ee77-498d-b5fa-677dcc0bd3a3 admin admin] PATH_INFO: `/v3/projects/56023d84553a4276b648a29f060d5963/users/fac389eb9c5e486caa9136e09b39585e/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.598754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80b960-ee77-498d-b5fa-677dcc0bd3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56023d84553a4276b648a29f060d5963, user_id=fac389eb9c5e486caa9136e09b39585e, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.601578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80b960-ee77-498d-b5fa-677dcc0bd3a3 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.627697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1382/2763] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:10 2022] PUT /identity/v3/projects/56023d84553a4276b648a29f060d5963/users/fac389eb9c5e486caa9136e09b39585e/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:10.642573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef26400-c305-4b62-9141-2ce7eef5a740 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.643001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef26400-c305-4b62-9141-2ce7eef5a740 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.644894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef26400-c305-4b62-9141-2ce7eef5a740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.663994 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80b960-ee77-498d-b5fa-677dcc0bd3a3 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.666467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3d80b960-ee77-498d-b5fa-677dcc0bd3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:10.699297 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6ef26400-c305-4b62-9141-2ce7eef5a740 None None] MFA Rules not processed for user `fac389eb9c5e486caa9136e09b39585e`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:10.701078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6ef26400-c305-4b62-9141-2ce7eef5a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:10.760626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80b960-ee77-498d-b5fa-677dcc0bd3a3 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.766044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.766988 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.767556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.767921 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] PATH_INFO: `/v3/projects/587879a4f8d74b159f8c7288276ca607` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.769670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] RBAC: Authorizing `identity:delete_project(project_id=587879a4f8d74b159f8c7288276ca607)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.771710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.775690 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1382/2764] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:10.779886 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef26400-c305-4b62-9141-2ce7eef5a740 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.791321 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef26400-c305-4b62-9141-2ce7eef5a740 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.798042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226c995-5df5-4480-b885-886ff2682775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.798760 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226c995-5df5-4480-b885-886ff2682775 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.799042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226c995-5df5-4480-b885-886ff2682775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.799561 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226c995-5df5-4480-b885-886ff2682775 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.800277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226c995-5df5-4480-b885-886ff2682775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.802213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226c995-5df5-4480-b885-886ff2682775 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.838927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1383/2765] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:10 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:30:10.840087 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226c995-5df5-4480-b885-886ff2682775 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.841829 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4226c995-5df5-4480-b885-886ff2682775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:10.846469 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587879a4f8d74b159f8c7288276ca607', 'request_id': 'req-bdaf96a1-91a4-4238-b8b2-3a4209decfca'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:10.855578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587879a4f8d74b159f8c7288276ca607', 'request_id': 'req-bdaf96a1-91a4-4238-b8b2-3a4209decfca'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:10.861309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587879a4f8d74b159f8c7288276ca607', 'request_id': 'req-bdaf96a1-91a4-4238-b8b2-3a4209decfca'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:10.876056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] The token cache is being invalidate because project 587879a4f8d74b159f8c7288276ca607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:10.876889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:10.886715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:10.889287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1383/2766] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:10 2022] DELETE /identity/v3/projects/587879a4f8d74b159f8c7288276ca607 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:10.889969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:10.891467 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:10.935588 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226c995-5df5-4480-b885-886ff2682775 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.941281 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d46182-dedd-411e-a5f7-b1f90efa5f4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:10.941936 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d46182-dedd-411e-a5f7-b1f90efa5f4d service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.942219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d46182-dedd-411e-a5f7-b1f90efa5f4d service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.942496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d46182-dedd-411e-a5f7-b1f90efa5f4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.944324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a1d46182-dedd-411e-a5f7-b1f90efa5f4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:10.953181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:10.958083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:10.958284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d46182-dedd-411e-a5f7-b1f90efa5f4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:10.964317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d46182-dedd-411e-a5f7-b1f90efa5f4d service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:10.964600 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf96a1-91a4-4238-b8b2-3a4209decfca admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:10.972950 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1384/2767] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:10.974452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c99ae2-3b3f-4747-9399-90edafeb05f4 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:10.974757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c99ae2-3b3f-4747-9399-90edafeb05f4 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:10.975019 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c99ae2-3b3f-4747-9399-90edafeb05f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:10.994201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-85c99ae2-3b3f-4747-9399-90edafeb05f4 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:10.995529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-85c99ae2-3b3f-4747-9399-90edafeb05f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:11.042394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97f4ef-b70f-4c5a-b3bf-3856e9e1579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.043112 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97f4ef-b70f-4c5a-b3bf-3856e9e1579e admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.043422 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97f4ef-b70f-4c5a-b3bf-3856e9e1579e admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.043700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97f4ef-b70f-4c5a-b3bf-3856e9e1579e admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.044408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97f4ef-b70f-4c5a-b3bf-3856e9e1579e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.046713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97f4ef-b70f-4c5a-b3bf-3856e9e1579e admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.064887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1385/2768] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:11.070127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1384/2769] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:10 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:11.073060 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16d243-8197-45f6-93d7-59f34619a6bb None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.073371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16d243-8197-45f6-93d7-59f34619a6bb None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.073647 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16d243-8197-45f6-93d7-59f34619a6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.075822 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97f4ef-b70f-4c5a-b3bf-3856e9e1579e admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.086803 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97f4ef-b70f-4c5a-b3bf-3856e9e1579e admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.091714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e799b69-fe19-4f6a-94dc-14b0290a6b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.092326 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e799b69-fe19-4f6a-94dc-14b0290a6b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.092610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e799b69-fe19-4f6a-94dc-14b0290a6b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.093045 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e799b69-fe19-4f6a-94dc-14b0290a6b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.093787 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e799b69-fe19-4f6a-94dc-14b0290a6b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.095173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-cd16d243-8197-45f6-93d7-59f34619a6bb None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:11.095474 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e799b69-fe19-4f6a-94dc-14b0290a6b42 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.096637 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cd16d243-8197-45f6-93d7-59f34619a6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:11.105960 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1386/2770] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:11.110074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1385/2771] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:11.115274 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e799b69-fe19-4f6a-94dc-14b0290a6b42 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.115487 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16d243-8197-45f6-93d7-59f34619a6bb None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.126039 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16d243-8197-45f6-93d7-59f34619a6bb None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.127650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e799b69-fe19-4f6a-94dc-14b0290a6b42 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.131759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7bb67-20e7-42a2-aa89-66230520ff4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.132441 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7bb67-20e7-42a2-aa89-66230520ff4c admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.132721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7bb67-20e7-42a2-aa89-66230520ff4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.132996 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7bb67-20e7-42a2-aa89-66230520ff4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.133188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688d6f1-5ecd-453c-bd4e-cadc31f3c4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.133731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7bb67-20e7-42a2-aa89-66230520ff4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:30:11.134133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7bb67-20e7-42a2-aa89-66230520ff4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.134744 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688d6f1-5ecd-453c-bd4e-cadc31f3c4f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.135170 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688d6f1-5ecd-453c-bd4e-cadc31f3c4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.135546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688d6f1-5ecd-453c-bd4e-cadc31f3c4f9 admin admin] PATH_INFO: `/v3/projects/a34db9ee4e29452ebc4a2599a53c64eb/users/0db4d0fdf52444659936ef8fc840c4ae/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.136005 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7bb67-20e7-42a2-aa89-66230520ff4c admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.151430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1387/2772] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:30:11.153568 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7bb67-20e7-42a2-aa89-66230520ff4c admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.157934 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0688d6f1-5ecd-453c-bd4e-cadc31f3c4f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34db9ee4e29452ebc4a2599a53c64eb, user_id=0db4d0fdf52444659936ef8fc840c4ae, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.160817 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0688d6f1-5ecd-453c-bd4e-cadc31f3c4f9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.164106 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7bb67-20e7-42a2-aa89-66230520ff4c admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.169933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c680c5-b456-4f8a-9fca-59864d655c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.170611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c680c5-b456-4f8a-9fca-59864d655c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.171238 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c680c5-b456-4f8a-9fca-59864d655c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.171902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c680c5-b456-4f8a-9fca-59864d655c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.172772 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c680c5-b456-4f8a-9fca-59864d655c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.177822 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c680c5-b456-4f8a-9fca-59864d655c97 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.190186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1386/2773] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:11 2022] PUT /identity/v3/projects/a34db9ee4e29452ebc4a2599a53c64eb/users/0db4d0fdf52444659936ef8fc840c4ae/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:11.197818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688d6f1-5ecd-453c-bd4e-cadc31f3c4f9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.214195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1388/2774] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:11 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:11.218814 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c680c5-b456-4f8a-9fca-59864d655c97 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.278099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688d6f1-5ecd-453c-bd4e-cadc31f3c4f9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.303833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c680c5-b456-4f8a-9fca-59864d655c97 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.310303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a003a-c1aa-45d4-b7c5-40cb20c93616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.310993 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a003a-c1aa-45d4-b7c5-40cb20c93616 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.311309 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a003a-c1aa-45d4-b7c5-40cb20c93616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.311589 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a003a-c1aa-45d4-b7c5-40cb20c93616 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.312290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a003a-c1aa-45d4-b7c5-40cb20c93616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.314307 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a003a-c1aa-45d4-b7c5-40cb20c93616 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.339290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1389/2775] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:11 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:11.342099 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a003a-c1aa-45d4-b7c5-40cb20c93616 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.349188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b27764-e80a-4794-8e7b-3480d9b91242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.349887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b27764-e80a-4794-8e7b-3480d9b91242 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.350166 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b27764-e80a-4794-8e7b-3480d9b91242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.350437 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b27764-e80a-4794-8e7b-3480d9b91242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.351396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b27764-e80a-4794-8e7b-3480d9b91242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.353031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b27764-e80a-4794-8e7b-3480d9b91242 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.360845 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a003a-c1aa-45d4-b7c5-40cb20c93616 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.367297 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1387/2776] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:30:11.368188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c4dfd-cadc-4c72-9c77-953b75554f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.368188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c4dfd-cadc-4c72-9c77-953b75554f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.368188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c4dfd-cadc-4c72-9c77-953b75554f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.368188 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c4dfd-cadc-4c72-9c77-953b75554f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.368497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c4dfd-cadc-4c72-9c77-953b75554f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.369518 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c4dfd-cadc-4c72-9c77-953b75554f32 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.375157 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b27764-e80a-4794-8e7b-3480d9b91242 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.384039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:11.384220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1390/2777] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:11.385116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c4dfd-cadc-4c72-9c77-953b75554f32 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.386493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b27764-e80a-4794-8e7b-3480d9b91242 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.392442 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a370a3-a483-453e-886b-bc1337c6ccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.393340 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a370a3-a483-453e-886b-bc1337c6ccfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.393617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a370a3-a483-453e-886b-bc1337c6ccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.393887 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a370a3-a483-453e-886b-bc1337c6ccfe admin admin] PATH_INFO: `/v3/projects/a34db9ee4e29452ebc4a2599a53c64eb/users/0db4d0fdf52444659936ef8fc840c4ae/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.394748 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c4dfd-cadc-4c72-9c77-953b75554f32 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.395900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a370a3-a483-453e-886b-bc1337c6ccfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34db9ee4e29452ebc4a2599a53c64eb, user_id=0db4d0fdf52444659936ef8fc840c4ae, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.398866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a370a3-a483-453e-886b-bc1337c6ccfe admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.399293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e2421-8c52-4cd5-8bc0-ce61188e7f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.399939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e2421-8c52-4cd5-8bc0-ce61188e7f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.400220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e2421-8c52-4cd5-8bc0-ce61188e7f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.400494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e2421-8c52-4cd5-8bc0-ce61188e7f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.401205 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e2421-8c52-4cd5-8bc0-ce61188e7f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.402773 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e2421-8c52-4cd5-8bc0-ce61188e7f8e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.409897 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1388/2778] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:11 2022] PUT /identity/v3/projects/a34db9ee4e29452ebc4a2599a53c64eb/users/0db4d0fdf52444659936ef8fc840c4ae/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:11.413947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1391/2779] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:11.417792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a370a3-a483-453e-886b-bc1337c6ccfe admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.419557 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab439774-bced-414b-b2b2-de36b084a175 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.420016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab439774-bced-414b-b2b2-de36b084a175 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.420311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab439774-bced-414b-b2b2-de36b084a175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.451127 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ab439774-bced-414b-b2b2-de36b084a175 None None] MFA Rules not processed for user `0db4d0fdf52444659936ef8fc840c4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:11.452391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ab439774-bced-414b-b2b2-de36b084a175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:11.461266 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:11.491610 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a370a3-a483-453e-886b-bc1337c6ccfe admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.496977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb2ae7-ae26-41a6-90ac-a6d287df7075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.497899 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb2ae7-ae26-41a6-90ac-a6d287df7075 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.499414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb2ae7-ae26-41a6-90ac-a6d287df7075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.499414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb2ae7-ae26-41a6-90ac-a6d287df7075 admin admin] PATH_INFO: `/v3/projects/f516a0cba82a44559ad5cf0107f26eac/users/2e1601a51462484b826514f29238f7e0/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.523178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb2ae7-ae26-41a6-90ac-a6d287df7075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f516a0cba82a44559ad5cf0107f26eac, user_id=2e1601a51462484b826514f29238f7e0, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.526086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb2ae7-ae26-41a6-90ac-a6d287df7075 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.539815 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1392/2780] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:11.544083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1389/2781] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:11 2022] PUT /identity/v3/projects/f516a0cba82a44559ad5cf0107f26eac/users/2e1601a51462484b826514f29238f7e0/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:11.553787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab439774-bced-414b-b2b2-de36b084a175 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.553787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ab439774-bced-414b-b2b2-de36b084a175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:11.555291 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289a749-7483-45fb-99da-922d8e035da6 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.555724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289a749-7483-45fb-99da-922d8e035da6 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.556007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289a749-7483-45fb-99da-922d8e035da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.585189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-2289a749-7483-45fb-99da-922d8e035da6 None None] MFA Rules not processed for user `2e1601a51462484b826514f29238f7e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:11.586438 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-2289a749-7483-45fb-99da-922d8e035da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:11.618416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab439774-bced-414b-b2b2-de36b084a175 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.665537 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1390/2782] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:11.668163 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9a320-f7d4-4fa8-8744-f713d3c2be97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.668819 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9a320-f7d4-4fa8-8744-f713d3c2be97 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.669384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9a320-f7d4-4fa8-8744-f713d3c2be97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.669778 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9a320-f7d4-4fa8-8744-f713d3c2be97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.669956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289a749-7483-45fb-99da-922d8e035da6 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.671857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-1aa9a320-f7d4-4fa8-8744-f713d3c2be97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:11.740022 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289a749-7483-45fb-99da-922d8e035da6 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.743151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9a320-f7d4-4fa8-8744-f713d3c2be97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.744917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9a320-f7d4-4fa8-8744-f713d3c2be97 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.745036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb63f1-593c-4daf-b411-f1bc67f7286c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.745792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb63f1-593c-4daf-b411-f1bc67f7286c admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.746082 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb63f1-593c-4daf-b411-f1bc67f7286c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.746359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb63f1-593c-4daf-b411-f1bc67f7286c admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.747056 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb63f1-593c-4daf-b411-f1bc67f7286c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.749031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb63f1-593c-4daf-b411-f1bc67f7286c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.758720 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:11.761927 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1393/2783] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:11.783322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1391/2784] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:11 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:30:11.785958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9a320-f7d4-4fa8-8744-f713d3c2be97 service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.797954 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9a320-f7d4-4fa8-8744-f713d3c2be97 service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.811572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:11.865285 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cb3fe-61be-4b73-bb53-b5ef04226754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.865982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cb3fe-61be-4b73-bb53-b5ef04226754 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.866493 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cb3fe-61be-4b73-bb53-b5ef04226754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.866782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cb3fe-61be-4b73-bb53-b5ef04226754 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.867615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cb3fe-61be-4b73-bb53-b5ef04226754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.869998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cb3fe-61be-4b73-bb53-b5ef04226754 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.897130 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1394/2785] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:11 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:30:11.898940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb63f1-593c-4daf-b411-f1bc67f7286c admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.909982 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb63f1-593c-4daf-b411-f1bc67f7286c admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.914923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7a7f2-7fc7-40a1-8f0d-c634ade417a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.915666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7a7f2-7fc7-40a1-8f0d-c634ade417a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.915960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7a7f2-7fc7-40a1-8f0d-c634ade417a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.916226 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7a7f2-7fc7-40a1-8f0d-c634ade417a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.916975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7a7f2-7fc7-40a1-8f0d-c634ade417a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.918617 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7a7f2-7fc7-40a1-8f0d-c634ade417a9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.933173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1392/2786] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:11.934460 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cb3fe-61be-4b73-bb53-b5ef04226754 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:11.946011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cb3fe-61be-4b73-bb53-b5ef04226754 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:11.952339 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a6580-f690-4041-9fcb-da2f532fc635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:11.953410 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a6580-f690-4041-9fcb-da2f532fc635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:11.953754 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a6580-f690-4041-9fcb-da2f532fc635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:11.954097 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a6580-f690-4041-9fcb-da2f532fc635 admin admin] PATH_INFO: `/v3/projects/71b9fc5645e447c9a26c82359719f4e6/users/7839f092088948bfb01bbbb77028a323/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:11.971842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a6580-f690-4041-9fcb-da2f532fc635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b9fc5645e447c9a26c82359719f4e6, user_id=7839f092088948bfb01bbbb77028a323, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:11.974533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a6580-f690-4041-9fcb-da2f532fc635 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:11.989825 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1395/2787] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:11 2022] PUT /identity/v3/projects/71b9fc5645e447c9a26c82359719f4e6/users/7839f092088948bfb01bbbb77028a323/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:11.993958 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7a7f2-7fc7-40a1-8f0d-c634ade417a9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:12.004559 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7a7f2-7fc7-40a1-8f0d-c634ade417a9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:12.069288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb19c4-04e6-4733-a50f-81b2a4711d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:12.069976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb19c4-04e6-4733-a50f-81b2a4711d31 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:12.070260 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb19c4-04e6-4733-a50f-81b2a4711d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:12.070531 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb19c4-04e6-4733-a50f-81b2a4711d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:12.071313 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb19c4-04e6-4733-a50f-81b2a4711d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:12.072977 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb19c4-04e6-4733-a50f-81b2a4711d31 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:12.085932 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1393/2788] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:30:12.090179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a6580-f690-4041-9fcb-da2f532fc635 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:12.102157 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a6580-f690-4041-9fcb-da2f532fc635 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:12.108482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d2dbe-f3bb-4839-a874-11a96a96a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:12.109438 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d2dbe-f3bb-4839-a874-11a96a96a9cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:12.109727 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d2dbe-f3bb-4839-a874-11a96a96a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:12.110003 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d2dbe-f3bb-4839-a874-11a96a96a9cc admin admin] PATH_INFO: `/v3/projects/71b9fc5645e447c9a26c82359719f4e6/users/7839f092088948bfb01bbbb77028a323/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:12.112150 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d2dbe-f3bb-4839-a874-11a96a96a9cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b9fc5645e447c9a26c82359719f4e6, user_id=7839f092088948bfb01bbbb77028a323, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:12.114899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d2dbe-f3bb-4839-a874-11a96a96a9cc admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:12.128035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1396/2789] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:12 2022] PUT /identity/v3/projects/71b9fc5645e447c9a26c82359719f4e6/users/7839f092088948bfb01bbbb77028a323/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:12.138540 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e6e3-fc90-4b12-bc21-24eeef51d8e1 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:12.139061 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e6e3-fc90-4b12-bc21-24eeef51d8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:12.139407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e6e3-fc90-4b12-bc21-24eeef51d8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:12.168188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-d5b0e6e3-fc90-4b12-bc21-24eeef51d8e1 None None] MFA Rules not processed for user `7839f092088948bfb01bbbb77028a323`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:12.169406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d5b0e6e3-fc90-4b12-bc21-24eeef51d8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:12.259411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1394/2790] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:12.276759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d2dbe-f3bb-4839-a874-11a96a96a9cc admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:12.343035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d2dbe-f3bb-4839-a874-11a96a96a9cc admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:12.347826 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e141a-182a-414f-ba6a-4f62468dae41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:12.348512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e141a-182a-414f-ba6a-4f62468dae41 service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:12.348792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e141a-182a-414f-ba6a-4f62468dae41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:12.349104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e141a-182a-414f-ba6a-4f62468dae41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:12.359491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e141a-182a-414f-ba6a-4f62468dae41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:12.361011 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e141a-182a-414f-ba6a-4f62468dae41 service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:12.371974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1397/2791] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:13.025505 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0e6e3-fc90-4b12-bc21-24eeef51d8e1 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:13.097672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:13.103016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0e6e3-fc90-4b12-bc21-24eeef51d8e1 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:13.110560 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:13.111489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:13.111770 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:13.112039 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] PATH_INFO: `/v3/users/2e1601a51462484b826514f29238f7e0` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:13.113240 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e1601a51462484b826514f29238f7e0)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:13.115205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:13.168155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e1601a51462484b826514f29238f7e0'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:13.175660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e1601a51462484b826514f29238f7e0'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:13.187053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e1601a51462484b826514f29238f7e0'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:13.187908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e1601a51462484b826514f29238f7e0'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:13.189048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e1601a51462484b826514f29238f7e0'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:13.212707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1395/2792] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:13 2022] DELETE /identity/v3/users/2e1601a51462484b826514f29238f7e0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:13.218089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f71830-f32e-46e3-83ae-b9bfd0b80631 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:13.218643 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f71830-f32e-46e3-83ae-b9bfd0b80631 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:13.218823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f71830-f32e-46e3-83ae-b9bfd0b80631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:13.240580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-39f71830-f32e-46e3-83ae-b9bfd0b80631 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:13.241792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-39f71830-f32e-46e3-83ae-b9bfd0b80631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:13.319738 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1398/2793] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:13.331036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:13.332743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:13.409975 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2f503-53a3-489a-a9ec-ef60c26caa95 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:13.413626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff7247-c4f5-4cf1-8b4d-19c8baf60e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:13.414275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff7247-c4f5-4cf1-8b4d-19c8baf60e7c service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:13.414551 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff7247-c4f5-4cf1-8b4d-19c8baf60e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:13.414819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff7247-c4f5-4cf1-8b4d-19c8baf60e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:13.424968 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff7247-c4f5-4cf1-8b4d-19c8baf60e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:13.426881 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff7247-c4f5-4cf1-8b4d-19c8baf60e7c service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:13.439211 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1396/2794] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:13.492873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f71830-f32e-46e3-83ae-b9bfd0b80631 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:13.494443 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-39f71830-f32e-46e3-83ae-b9bfd0b80631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:13.510678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:13.579736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f71830-f32e-46e3-83ae-b9bfd0b80631 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:13.593427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c1f21-ffc1-4050-972d-61981e467f7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:13.594220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c1f21-ffc1-4050-972d-61981e467f7d service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:13.594379 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c1f21-ffc1-4050-972d-61981e467f7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:13.594666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c1f21-ffc1-4050-972d-61981e467f7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:13.601254 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e59c1f21-ffc1-4050-972d-61981e467f7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:13.676667 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c1f21-ffc1-4050-972d-61981e467f7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:13.678134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c1f21-ffc1-4050-972d-61981e467f7d service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:13.711406 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1399/2795] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:13.831835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff7247-c4f5-4cf1-8b4d-19c8baf60e7c service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:13.833241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-73ff7247-c4f5-4cf1-8b4d-19c8baf60e7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:13.857016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff7247-c4f5-4cf1-8b4d-19c8baf60e7c service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:13.861599 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:13.862476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:13.862755 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:13.863064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] PATH_INFO: `/v3/projects/f516a0cba82a44559ad5cf0107f26eac` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:13.864481 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f516a0cba82a44559ad5cf0107f26eac)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:13.866656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:13.937418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f516a0cba82a44559ad5cf0107f26eac', 'request_id': 'req-45d0dbdd-8c1e-4dba-b51b-744071139fc9'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:13.944200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f516a0cba82a44559ad5cf0107f26eac', 'request_id': 'req-45d0dbdd-8c1e-4dba-b51b-744071139fc9'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:13.950792 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f516a0cba82a44559ad5cf0107f26eac', 'request_id': 'req-45d0dbdd-8c1e-4dba-b51b-744071139fc9'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:13.966576 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] The token cache is being invalidate because project f516a0cba82a44559ad5cf0107f26eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:13.966877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:13.968018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:13.973124 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1397/2796] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:13 2022] DELETE /identity/v3/projects/f516a0cba82a44559ad5cf0107f26eac => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:13.974768 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c1f21-ffc1-4050-972d-61981e467f7d service cinder] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:13.998397 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c1f21-ffc1-4050-972d-61981e467f7d service cinder] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:14.111578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:14.114145 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:14.114562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:14.115071 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] PATH_INFO: `/v3/users/7839f092088948bfb01bbbb77028a323` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:14.116823 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7839f092088948bfb01bbbb77028a323)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:14.120080 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:14.195311 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7839f092088948bfb01bbbb77028a323'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:14.208384 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7839f092088948bfb01bbbb77028a323'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:14.210132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7839f092088948bfb01bbbb77028a323'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:14.222024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7839f092088948bfb01bbbb77028a323'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:14.233943 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:14.245751 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7839f092088948bfb01bbbb77028a323'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:14.255326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1400/2797] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:13 2022] DELETE /identity/v3/users/7839f092088948bfb01bbbb77028a323 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:14.430703 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:14.432405 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:14.438719 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:14.507856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0dbdd-8c1e-4dba-b51b-744071139fc9 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:14.570613 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fbb9a-86ab-440a-888f-0410caa824d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:14.571367 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fbb9a-86ab-440a-888f-0410caa824d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:14.571718 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fbb9a-86ab-440a-888f-0410caa824d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:14.572095 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fbb9a-86ab-440a-888f-0410caa824d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:14.574517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-260fbb9a-86ab-440a-888f-0410caa824d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:14.596219 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:14.598009 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:14.653020 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fbb9a-86ab-440a-888f-0410caa824d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:14.655342 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fbb9a-86ab-440a-888f-0410caa824d3 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:14.668138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1398/2798] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 06:30:14.677922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05a87c-3431-49a8-8435-d00e08c88fef admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:14.682844 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:14.683796 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:14.684213 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:14.684565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] PATH_INFO: `/v3/projects/71b9fc5645e447c9a26c82359719f4e6` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:14.685946 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b9fc5645e447c9a26c82359719f4e6)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:14.687986 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:14.744181 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b9fc5645e447c9a26c82359719f4e6', 'request_id': 'req-f56e4d84-e13c-484c-8429-c9f590affd61'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:14.750293 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b9fc5645e447c9a26c82359719f4e6', 'request_id': 'req-f56e4d84-e13c-484c-8429-c9f590affd61'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:14.758961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b9fc5645e447c9a26c82359719f4e6', 'request_id': 'req-f56e4d84-e13c-484c-8429-c9f590affd61'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:14.774123 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] The token cache is being invalidate because project 71b9fc5645e447c9a26c82359719f4e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:14.774416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:14.775040 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f56e4d84-e13c-484c-8429-c9f590affd61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:14.776705 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1401/2799] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:14 2022] DELETE /identity/v3/projects/71b9fc5645e447c9a26c82359719f4e6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:14.781726 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f4667-a59c-4f27-9358-6b4aa2b941b9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:14.782023 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f4667-a59c-4f27-9358-6b4aa2b941b9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:14.782300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f4667-a59c-4f27-9358-6b4aa2b941b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:14.801476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-609f4667-a59c-4f27-9358-6b4aa2b941b9 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:14.802710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-609f4667-a59c-4f27-9358-6b4aa2b941b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:14.869924 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1399/2800] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:14.878454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ca072-d149-4037-b30d-d52294130d6e None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:14.878779 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ca072-d149-4037-b30d-d52294130d6e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:14.879047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ca072-d149-4037-b30d-d52294130d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:14.903093 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-798ca072-d149-4037-b30d-d52294130d6e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:14.904343 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-798ca072-d149-4037-b30d-d52294130d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:14.912645 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1402/2801] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:14.917237 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f4667-a59c-4f27-9358-6b4aa2b941b9 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:14.933879 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f4667-a59c-4f27-9358-6b4aa2b941b9 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:14.938839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c013e5-fc5c-4d30-ac40-2a0d95400529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:14.939608 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c013e5-fc5c-4d30-ac40-2a0d95400529 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:14.939893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c013e5-fc5c-4d30-ac40-2a0d95400529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:14.940162 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c013e5-fc5c-4d30-ac40-2a0d95400529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:14.940890 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c013e5-fc5c-4d30-ac40-2a0d95400529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:30:14.941196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c013e5-fc5c-4d30-ac40-2a0d95400529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:14.942839 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c013e5-fc5c-4d30-ac40-2a0d95400529 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:14.945400 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:14.958028 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1400/2802] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:30:14.969400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ca072-d149-4037-b30d-d52294130d6e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:14.978745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ca072-d149-4037-b30d-d52294130d6e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:14.984382 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeeddae-8c50-4fe7-8b52-af1b3a23ae66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:14.985275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeddae-8c50-4fe7-8b52-af1b3a23ae66 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:14.985757 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeddae-8c50-4fe7-8b52-af1b3a23ae66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:14.986134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeddae-8c50-4fe7-8b52-af1b3a23ae66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:14.986917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeeddae-8c50-4fe7-8b52-af1b3a23ae66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:14.989403 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeeddae-8c50-4fe7-8b52-af1b3a23ae66 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.024971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1403/2803] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:14 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:30:15.029192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c013e5-fc5c-4d30-ac40-2a0d95400529 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:15.041288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c013e5-fc5c-4d30-ac40-2a0d95400529 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:15.104018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d50bc-33fa-4252-bf05-9e52da57f169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:15.104704 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d50bc-33fa-4252-bf05-9e52da57f169 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.104992 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d50bc-33fa-4252-bf05-9e52da57f169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.105265 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d50bc-33fa-4252-bf05-9e52da57f169 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.106074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d50bc-33fa-4252-bf05-9e52da57f169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:15.108058 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d50bc-33fa-4252-bf05-9e52da57f169 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.131739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1401/2804] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:15 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:15.137118 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeeddae-8c50-4fe7-8b52-af1b3a23ae66 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:15.152180 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeeddae-8c50-4fe7-8b52-af1b3a23ae66 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:15.158994 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4adae-6a0e-49c2-9b69-daae717786a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:15.159720 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4adae-6a0e-49c2-9b69-daae717786a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.159974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4adae-6a0e-49c2-9b69-daae717786a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.160277 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4adae-6a0e-49c2-9b69-daae717786a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.161024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4adae-6a0e-49c2-9b69-daae717786a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:15.162656 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4adae-6a0e-49c2-9b69-daae717786a3 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.174300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1404/2805] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:15.178475 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d50bc-33fa-4252-bf05-9e52da57f169 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:15.189715 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d50bc-33fa-4252-bf05-9e52da57f169 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:15.195148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fbfbf-3431-4225-9917-9a436c254dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:15.195845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fbfbf-3431-4225-9917-9a436c254dac admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.196155 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fbfbf-3431-4225-9917-9a436c254dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.196434 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fbfbf-3431-4225-9917-9a436c254dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.197192 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fbfbf-3431-4225-9917-9a436c254dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:15.198800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fbfbf-3431-4225-9917-9a436c254dac admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.211040 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1402/2806] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:15.215013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4adae-6a0e-49c2-9b69-daae717786a3 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:15.225677 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4adae-6a0e-49c2-9b69-daae717786a3 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:15.231053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b9b7c-b063-4163-b55f-8ec63e5f79bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:15.232243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b9b7c-b063-4163-b55f-8ec63e5f79bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.232527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b9b7c-b063-4163-b55f-8ec63e5f79bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.232803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b9b7c-b063-4163-b55f-8ec63e5f79bc admin admin] PATH_INFO: `/v3/projects/9deba980af3f454aa394673e1b598aab/users/f022c978e3824a09a3439fa51844ace0/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.248505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b9b7c-b063-4163-b55f-8ec63e5f79bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9deba980af3f454aa394673e1b598aab, user_id=f022c978e3824a09a3439fa51844ace0, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:15.251270 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b9b7c-b063-4163-b55f-8ec63e5f79bc admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.268490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1405/2807] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:15 2022] PUT /identity/v3/projects/9deba980af3f454aa394673e1b598aab/users/f022c978e3824a09a3439fa51844ace0/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:15.279916 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe1bbe-83ef-4e90-8644-f8b6974691aa None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.280159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe1bbe-83ef-4e90-8644-f8b6974691aa None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.280423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe1bbe-83ef-4e90-8644-f8b6974691aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.308845 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-0efe1bbe-83ef-4e90-8644-f8b6974691aa None None] MFA Rules not processed for user `f022c978e3824a09a3439fa51844ace0`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:15.310086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-0efe1bbe-83ef-4e90-8644-f8b6974691aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:15.383054 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1403/2808] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:15.387699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b9b7c-b063-4163-b55f-8ec63e5f79bc admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:15.402034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:15.455218 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b9b7c-b063-4163-b55f-8ec63e5f79bc admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:15.460199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9accb5-997d-40e5-8807-d895be026dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:15.460871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9accb5-997d-40e5-8807-d895be026dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.461151 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9accb5-997d-40e5-8807-d895be026dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.461426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9accb5-997d-40e5-8807-d895be026dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.462128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9accb5-997d-40e5-8807-d895be026dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:15.464096 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9accb5-997d-40e5-8807-d895be026dd9 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.493848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1406/2809] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:15 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:30:15.498012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe1bbe-83ef-4e90-8644-f8b6974691aa None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:15.509287 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe1bbe-83ef-4e90-8644-f8b6974691aa None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:15.576191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d6c25-931d-4788-b1df-06ef19c5baf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:15.576888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d6c25-931d-4788-b1df-06ef19c5baf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.577168 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d6c25-931d-4788-b1df-06ef19c5baf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.577441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d6c25-931d-4788-b1df-06ef19c5baf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.578144 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d6c25-931d-4788-b1df-06ef19c5baf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:15.580334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d6c25-931d-4788-b1df-06ef19c5baf8 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.589106 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:15.606581 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1404/2810] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:15 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:15.610654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9accb5-997d-40e5-8807-d895be026dd9 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:15.621730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9accb5-997d-40e5-8807-d895be026dd9 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:15.627061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d220-0ef9-40de-b30a-b8060bd35da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:15.628100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d220-0ef9-40de-b30a-b8060bd35da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.628934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d220-0ef9-40de-b30a-b8060bd35da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.628934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d220-0ef9-40de-b30a-b8060bd35da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.629652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952d220-0ef9-40de-b30a-b8060bd35da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:15.633344 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952d220-0ef9-40de-b30a-b8060bd35da5 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.646687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1407/2811] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:15.651134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d6c25-931d-4788-b1df-06ef19c5baf8 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:15.661679 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d6c25-931d-4788-b1df-06ef19c5baf8 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:15.667008 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e461-9cfc-429d-adbf-d0023d76ec66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:15.667988 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e461-9cfc-429d-adbf-d0023d76ec66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.668278 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e461-9cfc-429d-adbf-d0023d76ec66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.668664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e461-9cfc-429d-adbf-d0023d76ec66 admin admin] PATH_INFO: `/v3/projects/da14c1c4a83d4f65b2f32ee747f2d121/users/36dfd9d006a346b0b83aef5dceabc6b7/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.688381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e461-9cfc-429d-adbf-d0023d76ec66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da14c1c4a83d4f65b2f32ee747f2d121, user_id=36dfd9d006a346b0b83aef5dceabc6b7, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:15.691364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e461-9cfc-429d-adbf-d0023d76ec66 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.705868 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1405/2812] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:15 2022] PUT /identity/v3/projects/da14c1c4a83d4f65b2f32ee747f2d121/users/36dfd9d006a346b0b83aef5dceabc6b7/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:15.710001 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d220-0ef9-40de-b30a-b8060bd35da5 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:15.721598 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d220-0ef9-40de-b30a-b8060bd35da5 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:15.786969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d37663-bf83-4b54-ba3f-a09f4d31b661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:15.787680 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d37663-bf83-4b54-ba3f-a09f4d31b661 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.789386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d37663-bf83-4b54-ba3f-a09f4d31b661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.789386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d37663-bf83-4b54-ba3f-a09f4d31b661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.789386 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d37663-bf83-4b54-ba3f-a09f4d31b661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:15.790775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d37663-bf83-4b54-ba3f-a09f4d31b661 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.802578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1408/2813] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:15.806863 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e461-9cfc-429d-adbf-d0023d76ec66 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:15.818630 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e461-9cfc-429d-adbf-d0023d76ec66 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:15.825885 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f11a3d-4852-4158-9710-2b135698b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:15.826919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f11a3d-4852-4158-9710-2b135698b266 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.827073 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f11a3d-4852-4158-9710-2b135698b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.827375 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f11a3d-4852-4158-9710-2b135698b266 admin admin] PATH_INFO: `/v3/projects/da14c1c4a83d4f65b2f32ee747f2d121/users/36dfd9d006a346b0b83aef5dceabc6b7/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.829359 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f11a3d-4852-4158-9710-2b135698b266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da14c1c4a83d4f65b2f32ee747f2d121, user_id=36dfd9d006a346b0b83aef5dceabc6b7, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:15.832486 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f11a3d-4852-4158-9710-2b135698b266 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:15.843795 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1406/2814] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:15 2022] PUT /identity/v3/projects/da14c1c4a83d4f65b2f32ee747f2d121/users/36dfd9d006a346b0b83aef5dceabc6b7/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:15.854179 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c63e9c-d730-4938-bb2d-78b51f91d01d None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:15.854456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c63e9c-d730-4938-bb2d-78b51f91d01d None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:15.854725 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c63e9c-d730-4938-bb2d-78b51f91d01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:15.888852 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-49c63e9c-d730-4938-bb2d-78b51f91d01d None None] MFA Rules not processed for user `36dfd9d006a346b0b83aef5dceabc6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:15.891116 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-49c63e9c-d730-4938-bb2d-78b51f91d01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:16.057572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1409/2815] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:16.076081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f11a3d-4852-4158-9710-2b135698b266 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:16.078148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-21f11a3d-4852-4158-9710-2b135698b266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:16.193861 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f11a3d-4852-4158-9710-2b135698b266 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:16.199062 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15349dc-7f0b-4edd-b155-32d2b537bc56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:16.199904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15349dc-7f0b-4edd-b155-32d2b537bc56 service cinder] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:16.200262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15349dc-7f0b-4edd-b155-32d2b537bc56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:16.200743 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15349dc-7f0b-4edd-b155-32d2b537bc56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:16.217083 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15349dc-7f0b-4edd-b155-32d2b537bc56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:16.219063 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15349dc-7f0b-4edd-b155-32d2b537bc56 service cinder] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:16.233368 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1407/2816] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:16.649533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c63e9c-d730-4938-bb2d-78b51f91d01d None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:16.658613 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:16.716189 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c63e9c-d730-4938-bb2d-78b51f91d01d None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:16.721109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:16.722275 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:16.722381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:16.722611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] PATH_INFO: `/v3/users/f022c978e3824a09a3439fa51844ace0` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:16.724302 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f022c978e3824a09a3439fa51844ace0)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:16.727867 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:16.782120 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f022c978e3824a09a3439fa51844ace0'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:16.790053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f022c978e3824a09a3439fa51844ace0'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:16.790832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f022c978e3824a09a3439fa51844ace0'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:16.801971 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f022c978e3824a09a3439fa51844ace0'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:16.815077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f022c978e3824a09a3439fa51844ace0'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:16.821166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1410/2817] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:16 2022] DELETE /identity/v3/users/f022c978e3824a09a3439fa51844ace0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:16.825832 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470381-abd0-414b-851d-ae2d4c9b04da None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:16.826330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470381-abd0-414b-851d-ae2d4c9b04da None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:16.826330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470381-abd0-414b-851d-ae2d4c9b04da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:16.845652 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ef470381-abd0-414b-851d-ae2d4c9b04da None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:16.846920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ef470381-abd0-414b-851d-ae2d4c9b04da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:16.912935 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1408/2818] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:16.924035 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:16.925794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:16.995841 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372522f9-e06b-4b44-8f0b-af98b2ed9f08 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:17.000350 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0281762-8817-4e94-9ac8-60b66a0496f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:17.001081 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0281762-8817-4e94-9ac8-60b66a0496f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:17.001430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0281762-8817-4e94-9ac8-60b66a0496f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:17.001827 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0281762-8817-4e94-9ac8-60b66a0496f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:17.015529 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0281762-8817-4e94-9ac8-60b66a0496f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:17.017373 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0281762-8817-4e94-9ac8-60b66a0496f2 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:17.027222 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1411/2819] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:17.326064 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef470381-abd0-414b-851d-ae2d4c9b04da None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:17.328638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ef470381-abd0-414b-851d-ae2d4c9b04da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:17.344759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef470381-abd0-414b-851d-ae2d4c9b04da None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:17.357066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:17.357944 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:17.358227 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:17.358504 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] PATH_INFO: `/v3/projects/9deba980af3f454aa394673e1b598aab` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:17.359898 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9deba980af3f454aa394673e1b598aab)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:17.361950 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:17.419782 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9deba980af3f454aa394673e1b598aab', 'request_id': 'req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:17.426350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9deba980af3f454aa394673e1b598aab', 'request_id': 'req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:17.428181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:17.433385 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9deba980af3f454aa394673e1b598aab', 'request_id': 'req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:17.447423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] The token cache is being invalidate because project 9deba980af3f454aa394673e1b598aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:17.447760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:17.448372 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-41dff29b-09bf-4d5f-bed1-61ae2d5dddd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:17.450035 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1409/2820] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:17 2022] DELETE /identity/v3/projects/9deba980af3f454aa394673e1b598aab => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:17.454064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0281762-8817-4e94-9ac8-60b66a0496f2 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:17.467059 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0281762-8817-4e94-9ac8-60b66a0496f2 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:17.493173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:17.530114 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:17.531083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:17.531433 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:17.531715 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] PATH_INFO: `/v3/users/36dfd9d006a346b0b83aef5dceabc6b7` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:17.532984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=36dfd9d006a346b0b83aef5dceabc6b7)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:17.535036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:17.585995 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36dfd9d006a346b0b83aef5dceabc6b7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:17.593580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36dfd9d006a346b0b83aef5dceabc6b7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:17.594278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36dfd9d006a346b0b83aef5dceabc6b7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:17.611875 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36dfd9d006a346b0b83aef5dceabc6b7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:17.633085 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36dfd9d006a346b0b83aef5dceabc6b7'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:17.639510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1412/2821] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:17 2022] DELETE /identity/v3/users/36dfd9d006a346b0b83aef5dceabc6b7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:17.792333 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046528c0-797e-4c0b-9718-71cc1638ba68 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:17.792624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046528c0-797e-4c0b-9718-71cc1638ba68 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:17.792900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046528c0-797e-4c0b-9718-71cc1638ba68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:17.854078 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-046528c0-797e-4c0b-9718-71cc1638ba68 None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:17.855110 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-046528c0-797e-4c0b-9718-71cc1638ba68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:17.923334 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:17.924912 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:17.931414 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1410/2822] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 06:30:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:17.992933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046528c0-797e-4c0b-9718-71cc1638ba68 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:17.994509 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-046528c0-797e-4c0b-9718-71cc1638ba68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:17.998617 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e294e8-33b3-456a-ad2d-0c3c8cc665ee admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:18.005538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:18.007426 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.007785 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.008134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] PATH_INFO: `/v3/projects/da14c1c4a83d4f65b2f32ee747f2d121` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.009409 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da14c1c4a83d4f65b2f32ee747f2d121)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:18.011621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:18.062083 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da14c1c4a83d4f65b2f32ee747f2d121', 'request_id': 'req-621f3584-da17-41b2-8375-455b0ad230a8'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:18.063304 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:18.068866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da14c1c4a83d4f65b2f32ee747f2d121', 'request_id': 'req-621f3584-da17-41b2-8375-455b0ad230a8'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:18.076896 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da14c1c4a83d4f65b2f32ee747f2d121', 'request_id': 'req-621f3584-da17-41b2-8375-455b0ad230a8'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:18.095630 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] The token cache is being invalidate because project da14c1c4a83d4f65b2f32ee747f2d121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:18.096048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:18.096176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046528c0-797e-4c0b-9718-71cc1638ba68 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:18.096899 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-621f3584-da17-41b2-8375-455b0ad230a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:18.098770 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1413/2823] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:17 2022] DELETE /identity/v3/projects/da14c1c4a83d4f65b2f32ee747f2d121 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:18.104274 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c4c51-bc1f-4346-ae86-343b081aec20 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.104958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c4c51-bc1f-4346-ae86-343b081aec20 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.105065 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c4c51-bc1f-4346-ae86-343b081aec20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.125280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-af5c4c51-bc1f-4346-ae86-343b081aec20 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:18.126562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-af5c4c51-bc1f-4346-ae86-343b081aec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:18.128133 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:18.168127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7d86b-7513-4df9-93c3-d5152f384eb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:18.169018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7d86b-7513-4df9-93c3-d5152f384eb0 service swift] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.169376 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7d86b-7513-4df9-93c3-d5152f384eb0 service swift] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.169694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7d86b-7513-4df9-93c3-d5152f384eb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.172937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-29a7d86b-7513-4df9-93c3-d5152f384eb0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:18.189830 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1414/2824] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:18.199303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44857228-c9c0-45e5-be46-53f8aa6745ea None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.199303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44857228-c9c0-45e5-be46-53f8aa6745ea None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.199678 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44857228-c9c0-45e5-be46-53f8aa6745ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.221848 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-44857228-c9c0-45e5-be46-53f8aa6745ea None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:18.223202 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-44857228-c9c0-45e5-be46-53f8aa6745ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:18.231607 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1415/2825] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:18.236042 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44857228-c9c0-45e5-be46-53f8aa6745ea None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:18.245675 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44857228-c9c0-45e5-be46-53f8aa6745ea None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:18.250283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4c463-18a2-4fc6-a7b7-e544f5b77cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:18.250957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4c463-18a2-4fc6-a7b7-e544f5b77cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.251262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4c463-18a2-4fc6-a7b7-e544f5b77cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.251595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4c463-18a2-4fc6-a7b7-e544f5b77cf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.252685 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4c463-18a2-4fc6-a7b7-e544f5b77cf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80128) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:30:18.253015 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4c463-18a2-4fc6-a7b7-e544f5b77cf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:18.254444 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7d86b-7513-4df9-93c3-d5152f384eb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:18.254721 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4c463-18a2-4fc6-a7b7-e544f5b77cf7 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:18.256190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7d86b-7513-4df9-93c3-d5152f384eb0 service swift] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:18.265710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1411/2826] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 06:30:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 06:30:18.271237 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1416/2827] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 06:30:18.273038 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7d86b-7513-4df9-93c3-d5152f384eb0 service swift] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:18.284458 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7d86b-7513-4df9-93c3-d5152f384eb0 service swift] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:18.289188 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622765c-b29c-4c7d-949a-5c2237da5770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:18.289850 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622765c-b29c-4c7d-949a-5c2237da5770 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.290128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622765c-b29c-4c7d-949a-5c2237da5770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.290396 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622765c-b29c-4c7d-949a-5c2237da5770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.291080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622765c-b29c-4c7d-949a-5c2237da5770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:18.293016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622765c-b29c-4c7d-949a-5c2237da5770 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:18.326912 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1412/2828] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:18 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:18.328100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4c463-18a2-4fc6-a7b7-e544f5b77cf7 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:18.338765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4c463-18a2-4fc6-a7b7-e544f5b77cf7 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:18.399139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166bc4-65b0-4200-af36-8bf303ecaff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:18.400024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166bc4-65b0-4200-af36-8bf303ecaff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.400317 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166bc4-65b0-4200-af36-8bf303ecaff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.400591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166bc4-65b0-4200-af36-8bf303ecaff1 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.401618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df166bc4-65b0-4200-af36-8bf303ecaff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:18.403566 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df166bc4-65b0-4200-af36-8bf303ecaff1 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:18.424926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1417/2829] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:18 2022] POST /identity/v3/users => generated 280 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:18.429776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622765c-b29c-4c7d-949a-5c2237da5770 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:18.440007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622765c-b29c-4c7d-949a-5c2237da5770 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:18.444694 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18af72-3acf-481d-80cd-16985721886a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:18.445360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18af72-3acf-481d-80cd-16985721886a admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.445638 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18af72-3acf-481d-80cd-16985721886a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.446241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18af72-3acf-481d-80cd-16985721886a admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.446663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18af72-3acf-481d-80cd-16985721886a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:18.448290 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18af72-3acf-481d-80cd-16985721886a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:18.459681 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1413/2830] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:18.463635 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166bc4-65b0-4200-af36-8bf303ecaff1 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:18.473730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166bc4-65b0-4200-af36-8bf303ecaff1 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:18.479109 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14c8af-b136-4abd-b9fa-f25e5506781f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:18.479513 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14c8af-b136-4abd-b9fa-f25e5506781f admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.479791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14c8af-b136-4abd-b9fa-f25e5506781f admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.480036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14c8af-b136-4abd-b9fa-f25e5506781f admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.480933 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14c8af-b136-4abd-b9fa-f25e5506781f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:18.482468 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14c8af-b136-4abd-b9fa-f25e5506781f admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:18.493454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1418/2831] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:18.497902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18af72-3acf-481d-80cd-16985721886a admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:18.510974 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18af72-3acf-481d-80cd-16985721886a admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:18.516589 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986bf96-5acb-49f8-adcb-f17c5277200b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:18.520360 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986bf96-5acb-49f8-adcb-f17c5277200b admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.520651 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986bf96-5acb-49f8-adcb-f17c5277200b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.520931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986bf96-5acb-49f8-adcb-f17c5277200b admin admin] PATH_INFO: `/v3/projects/eed9ab89f426411b88aaa8c2334b5b99/users/ce9bd26f0fe5428bb690729fe37846ec/roles/d0a32856510a480a897677657ace20de` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.537750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986bf96-5acb-49f8-adcb-f17c5277200b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed9ab89f426411b88aaa8c2334b5b99, user_id=ce9bd26f0fe5428bb690729fe37846ec, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:18.541209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986bf96-5acb-49f8-adcb-f17c5277200b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:18.562941 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1414/2832] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:18 2022] PUT /identity/v3/projects/eed9ab89f426411b88aaa8c2334b5b99/users/ce9bd26f0fe5428bb690729fe37846ec/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:18.574376 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3577a7-6f6d-45ea-a98f-ebc207901ca2 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.574681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3577a7-6f6d-45ea-a98f-ebc207901ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.576295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3577a7-6f6d-45ea-a98f-ebc207901ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.611551 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-0d3577a7-6f6d-45ea-a98f-ebc207901ca2 None None] MFA Rules not processed for user `ce9bd26f0fe5428bb690729fe37846ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:18.613048 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0d3577a7-6f6d-45ea-a98f-ebc207901ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:18.654315 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:18.689939 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986bf96-5acb-49f8-adcb-f17c5277200b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:18.691800 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-8986bf96-5acb-49f8-adcb-f17c5277200b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:18.733100 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1419/2833] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:18 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:18.772414 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3577a7-6f6d-45ea-a98f-ebc207901ca2 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:18.775913 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0d3577a7-6f6d-45ea-a98f-ebc207901ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:18.785519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986bf96-5acb-49f8-adcb-f17c5277200b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:18.791888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:18.793304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.793981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.794436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] PATH_INFO: `/v3/users/a3c78a52296e4ccbbe77eb433aa558a6` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.795967 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c78a52296e4ccbbe77eb433aa558a6)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:18.798142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:18.817404 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:18.867032 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3577a7-6f6d-45ea-a98f-ebc207901ca2 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:18.871505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f574e-3ea4-45b8-ac61-cc125e703a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:18.872730 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f574e-3ea4-45b8-ac61-cc125e703a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.873126 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f574e-3ea4-45b8-ac61-cc125e703a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.873412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f574e-3ea4-45b8-ac61-cc125e703a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.877489 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c78a52296e4ccbbe77eb433aa558a6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:18.883574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f574e-3ea4-45b8-ac61-cc125e703a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:18.885094 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c78a52296e4ccbbe77eb433aa558a6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:18.885228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f574e-3ea4-45b8-ac61-cc125e703a0a service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:18.894259 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c78a52296e4ccbbe77eb433aa558a6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:18.894941 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1420/2834] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:18 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 06:30:18.895109 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c78a52296e4ccbbe77eb433aa558a6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:18.895665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c78a52296e4ccbbe77eb433aa558a6'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:18.904223 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ffbe3-b2ee-4397-9a8a-10f743cf92c3 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.904507 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ffbe3-b2ee-4397-9a8a-10f743cf92c3 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.904782 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ffbe3-b2ee-4397-9a8a-10f743cf92c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.918498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1415/2835] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:18 2022] DELETE /identity/v3/users/a3c78a52296e4ccbbe77eb433aa558a6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:18.919623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:18.921043 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:18.925140 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-369ffbe3-b2ee-4397-9a8a-10f743cf92c3 None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:18.926506 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-369ffbe3-b2ee-4397-9a8a-10f743cf92c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:18.986688 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1421/2836] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 4 06:30:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:18.989276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b2869-9d22-4541-ae45-666d35866cf1 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:18.989580 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b2869-9d22-4541-ae45-666d35866cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:18.989871 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b2869-9d22-4541-ae45-666d35866cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:18.994443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a0ec6-4912-43d1-89cc-b27153dba70b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:19.008099 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:19.009242 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:19.009526 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:19.009799 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] PATH_INFO: `/v3/users/ce9bd26f0fe5428bb690729fe37846ec` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:19.011212 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9bd26f0fe5428bb690729fe37846ec)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:19.013215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:19.016303 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-cc2b2869-9d22-4541-ae45-666d35866cf1 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:19.018330 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cc2b2869-9d22-4541-ae45-666d35866cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.027338 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1422/2837] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:19.028161 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b2869-9d22-4541-ae45-666d35866cf1 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:19.029862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cc2b2869-9d22-4541-ae45-666d35866cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.076330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9bd26f0fe5428bb690729fe37846ec'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:19.083697 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9bd26f0fe5428bb690729fe37846ec'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:19.093657 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9bd26f0fe5428bb690729fe37846ec'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:19.094461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9bd26f0fe5428bb690729fe37846ec'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:19.095654 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9bd26f0fe5428bb690729fe37846ec'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:19.116774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1416/2838] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:18 2022] DELETE /identity/v3/users/ce9bd26f0fe5428bb690729fe37846ec => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:19.117237 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b2869-9d22-4541-ae45-666d35866cf1 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:19.118012 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:19.120660 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.127925 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=100 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:19.136902 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:19.192904 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4909227-9de3-4d6f-8b1a-6f4837514dc6 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:19.196826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd0c1a-6b9f-49ee-bc1a-07542d6a59b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:19.197498 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd0c1a-6b9f-49ee-bc1a-07542d6a59b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:19.197779 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd0c1a-6b9f-49ee-bc1a-07542d6a59b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:19.198053 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd0c1a-6b9f-49ee-bc1a-07542d6a59b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:19.199991 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cdfd0c1a-6b9f-49ee-bc1a-07542d6a59b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.204907 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6670c80-5785-4adf-ac81-09f41bd9d348 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:19.206016 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6670c80-5785-4adf-ac81-09f41bd9d348 service swift] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:19.206491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6670c80-5785-4adf-ac81-09f41bd9d348 service swift] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:19.206866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6670c80-5785-4adf-ac81-09f41bd9d348 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:19.208856 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e6670c80-5785-4adf-ac81-09f41bd9d348 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.270939 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6670c80-5785-4adf-ac81-09f41bd9d348 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:19.272620 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6670c80-5785-4adf-ac81-09f41bd9d348 service swift] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:19.276084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd0c1a-6b9f-49ee-bc1a-07542d6a59b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:19.277644 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd0c1a-6b9f-49ee-bc1a-07542d6a59b2 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:19.281712 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1423/2839] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 06:30:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 06:30:19.284104 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf07ae9-d1df-4517-a7a3-9c83972d8948 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:19.284392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf07ae9-d1df-4517-a7a3-9c83972d8948 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:19.284670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf07ae9-d1df-4517-a7a3-9c83972d8948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:19.293036 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1417/2840] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:19.309710 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-8cf07ae9-d1df-4517-a7a3-9c83972d8948 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:19.311322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8cf07ae9-d1df-4517-a7a3-9c83972d8948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.321454 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1424/2841] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:19.332350 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd0c1a-6b9f-49ee-bc1a-07542d6a59b2 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:19.335235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cdfd0c1a-6b9f-49ee-bc1a-07542d6a59b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.350445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd0c1a-6b9f-49ee-bc1a-07542d6a59b2 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:19.355901 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e593af5-d9c4-456c-aeff-e950ad9454a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:19.356920 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593af5-d9c4-456c-aeff-e950ad9454a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:19.357222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593af5-d9c4-456c-aeff-e950ad9454a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:19.357478 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593af5-d9c4-456c-aeff-e950ad9454a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:19.371999 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e593af5-d9c4-456c-aeff-e950ad9454a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:19.374585 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e593af5-d9c4-456c-aeff-e950ad9454a2 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:19.387867 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1418/2842] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:19.586484 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fbc3f-0e21-4bee-8866-2c948f05f94b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:19.586764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fbc3f-0e21-4bee-8866-2c948f05f94b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:19.587036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fbc3f-0e21-4bee-8866-2c948f05f94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:19.641292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-ea4fbc3f-0e21-4bee-8866-2c948f05f94b None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:19.642756 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ea4fbc3f-0e21-4bee-8866-2c948f05f94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.651999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1425/2843] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 06:30:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:19.666872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e593af5-d9c4-456c-aeff-e950ad9454a2 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:19.668406 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-3e593af5-d9c4-456c-aeff-e950ad9454a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.683575 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fbc3f-0e21-4bee-8866-2c948f05f94b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:19.685942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-ea4fbc3f-0e21-4bee-8866-2c948f05f94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.686256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e593af5-d9c4-456c-aeff-e950ad9454a2 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:19.693044 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7013d-ad21-43bc-93a1-e518ae1793e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:19.693727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7013d-ad21-43bc-93a1-e518ae1793e6 service swift] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:19.694010 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7013d-ad21-43bc-93a1-e518ae1793e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:19.694306 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7013d-ad21-43bc-93a1-e518ae1793e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:19.711533 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c7013d-ad21-43bc-93a1-e518ae1793e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:19.713320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c7013d-ad21-43bc-93a1-e518ae1793e6 service swift] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:19.729440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fbc3f-0e21-4bee-8866-2c948f05f94b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:19.732817 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1419/2844] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 06:30:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 06:30:19.738440 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:19.739439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:19.739745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:19.740050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] PATH_INFO: `/v3/projects/4167e0d6bb6648f5a161edd2bb0c98fa` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:19.741392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4167e0d6bb6648f5a161edd2bb0c98fa)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:19.744288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:19.801336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7013d-ad21-43bc-93a1-e518ae1793e6 service swift] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:19.804970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d7c7013d-ad21-43bc-93a1-e518ae1793e6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.836656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7013d-ad21-43bc-93a1-e518ae1793e6 service swift] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:19.850062 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4167e0d6bb6648f5a161edd2bb0c98fa', 'request_id': 'req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:19.860952 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4167e0d6bb6648f5a161edd2bb0c98fa', 'request_id': 'req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:19.876257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4167e0d6bb6648f5a161edd2bb0c98fa', 'request_id': 'req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:19.903618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] The token cache is being invalidate because project 4167e0d6bb6648f5a161edd2bb0c98fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:19.904053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:19.904885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:19.906974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1426/2845] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:19 2022] DELETE /identity/v3/projects/4167e0d6bb6648f5a161edd2bb0c98fa => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:19.911010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:19.917284 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:19.926714 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:19.927663 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:19.927922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:19.928199 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] PATH_INFO: `/v3/projects/eed9ab89f426411b88aaa8c2334b5b99` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:19.929764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed9ab89f426411b88aaa8c2334b5b99)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:19.931727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:19.943817 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a966bb-4e51-4b75-bf21-3ca95faf52e4 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:19.950193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:19.951087 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:19.951445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:19.951775 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] PATH_INFO: `/v3/users/fac389eb9c5e486caa9136e09b39585e` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:19.953076 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac389eb9c5e486caa9136e09b39585e)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:19.955006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:19.976548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:19.998361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:20.033915 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac389eb9c5e486caa9136e09b39585e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.034691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed9ab89f426411b88aaa8c2334b5b99', 'request_id': 'req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.044903 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac389eb9c5e486caa9136e09b39585e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.046138 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac389eb9c5e486caa9136e09b39585e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.048357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed9ab89f426411b88aaa8c2334b5b99', 'request_id': 'req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.057000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed9ab89f426411b88aaa8c2334b5b99', 'request_id': 'req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.059469 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac389eb9c5e486caa9136e09b39585e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.078256 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] The token cache is being invalidate because project eed9ab89f426411b88aaa8c2334b5b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:20.078527 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.079178 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-e06b7c27-5e8a-4e4c-b3f8-fbc3892791fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.080940 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1420/2846] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:19 2022] DELETE /identity/v3/projects/eed9ab89f426411b88aaa8c2334b5b99 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:20.083558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-bb0c258a-9dd3-479c-816b-0b203376311e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac389eb9c5e486caa9136e09b39585e'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.087309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5124f6-a5a1-4cce-9696-21ef4fa8d590 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.087760 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5124f6-a5a1-4cce-9696-21ef4fa8d590 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.088100 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5124f6-a5a1-4cce-9696-21ef4fa8d590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.094851 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1427/2847] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:19 2022] DELETE /identity/v3/users/fac389eb9c5e486caa9136e09b39585e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:20.112311 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ec5124f6-a5a1-4cce-9696-21ef4fa8d590 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:20.113525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ec5124f6-a5a1-4cce-9696-21ef4fa8d590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:20.192137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2f1bf-1c68-4a6a-82b1-88544eb8ce9c None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.196428 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2f1bf-1c68-4a6a-82b1-88544eb8ce9c None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.197167 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2f1bf-1c68-4a6a-82b1-88544eb8ce9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.232866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-cfb2f1bf-1c68-4a6a-82b1-88544eb8ce9c None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:20.234322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cfb2f1bf-1c68-4a6a-82b1-88544eb8ce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:20.244410 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1421/2848] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:20.245976 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934618f8-ba2c-4d5f-a7e8-40268c6c5d09 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.246369 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934618f8-ba2c-4d5f-a7e8-40268c6c5d09 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.246710 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934618f8-ba2c-4d5f-a7e8-40268c6c5d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.271783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-934618f8-ba2c-4d5f-a7e8-40268c6c5d09 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:20.273210 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-934618f8-ba2c-4d5f-a7e8-40268c6c5d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:20.303558 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1422/2849] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:20.308354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934618f8-ba2c-4d5f-a7e8-40268c6c5d09 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:20.324480 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934618f8-ba2c-4d5f-a7e8-40268c6c5d09 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:20.333361 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d88ad7-81dc-4302-8453-130a4e121b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:20.334042 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d88ad7-81dc-4302-8453-130a4e121b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.334391 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d88ad7-81dc-4302-8453-130a4e121b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.334670 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d88ad7-81dc-4302-8453-130a4e121b10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.335445 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d88ad7-81dc-4302-8453-130a4e121b10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:30:20.335756 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d88ad7-81dc-4302-8453-130a4e121b10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:20.337401 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d88ad7-81dc-4302-8453-130a4e121b10 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:20.352232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1423/2850] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:30:20.356668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d88ad7-81dc-4302-8453-130a4e121b10 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:20.357608 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1428/2851] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 4 06:30:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:20.373745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2f1bf-1c68-4a6a-82b1-88544eb8ce9c None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:20.376412 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-cfb2f1bf-1c68-4a6a-82b1-88544eb8ce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:20.379462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d88ad7-81dc-4302-8453-130a4e121b10 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:20.385459 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837dc40-6563-46e3-863c-be0f94db7003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:20.386195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837dc40-6563-46e3-863c-be0f94db7003 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.386611 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837dc40-6563-46e3-863c-be0f94db7003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.386961 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837dc40-6563-46e3-863c-be0f94db7003 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.387927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837dc40-6563-46e3-863c-be0f94db7003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:20.390587 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837dc40-6563-46e3-863c-be0f94db7003 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:20.441182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1424/2852] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:20 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:20.446407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837dc40-6563-46e3-863c-be0f94db7003 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:20.490847 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2f1bf-1c68-4a6a-82b1-88544eb8ce9c None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:20.543923 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837dc40-6563-46e3-863c-be0f94db7003 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:20.545492 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:20.548835 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1749843-b26e-4d49-b4b7-7e4ccafaf9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:20.549544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1749843-b26e-4d49-b4b7-7e4ccafaf9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.549819 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1749843-b26e-4d49-b4b7-7e4ccafaf9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.550093 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1749843-b26e-4d49-b4b7-7e4ccafaf9b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.550789 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1749843-b26e-4d49-b4b7-7e4ccafaf9b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:20.552768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1749843-b26e-4d49-b4b7-7e4ccafaf9b2 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:20.569525 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7783ad-136a-4fdc-9138-a95bbec226ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:20.570195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7783ad-136a-4fdc-9138-a95bbec226ff service swift] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.570479 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7783ad-136a-4fdc-9138-a95bbec226ff service swift] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.570788 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7783ad-136a-4fdc-9138-a95bbec226ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.574214 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1425/2853] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:20 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:30:20.575563 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1749843-b26e-4d49-b4b7-7e4ccafaf9b2 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:20.580566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e1749843-b26e-4d49-b4b7-7e4ccafaf9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:20.592566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1749843-b26e-4d49-b4b7-7e4ccafaf9b2 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:20.597517 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:20.598387 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.598675 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.598953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] PATH_INFO: `/v3/projects/56023d84553a4276b648a29f060d5963` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.600347 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56023d84553a4276b648a29f060d5963)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:20.602262 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:20.610668 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:20.631191 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7783ad-136a-4fdc-9138-a95bbec226ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:20.633075 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7783ad-136a-4fdc-9138-a95bbec226ff service swift] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:20.642024 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1429/2854] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 06:30:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 06:30:20.642863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7783ad-136a-4fdc-9138-a95bbec226ff service swift] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:20.653394 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7783ad-136a-4fdc-9138-a95bbec226ff service swift] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:20.659596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56023d84553a4276b648a29f060d5963', 'request_id': 'req-b930a567-81c0-4fb3-b02b-df4771fcef83'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.668750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56023d84553a4276b648a29f060d5963', 'request_id': 'req-b930a567-81c0-4fb3-b02b-df4771fcef83'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.677145 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56023d84553a4276b648a29f060d5963', 'request_id': 'req-b930a567-81c0-4fb3-b02b-df4771fcef83'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.696009 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] The token cache is being invalidate because project 56023d84553a4276b648a29f060d5963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:20.696872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.697173 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.701891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1426/2855] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:20 2022] DELETE /identity/v3/projects/56023d84553a4276b648a29f060d5963 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:20.702750 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:20.704186 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:20.722391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c767b-fc23-4f47-8a66-21dd8916ef87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:20.723078 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c767b-fc23-4f47-8a66-21dd8916ef87 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.723458 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c767b-fc23-4f47-8a66-21dd8916ef87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.724261 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c767b-fc23-4f47-8a66-21dd8916ef87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.724842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c767b-fc23-4f47-8a66-21dd8916ef87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:20.726862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c767b-fc23-4f47-8a66-21dd8916ef87 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:20.738729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1430/2856] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:20.743458 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c767b-fc23-4f47-8a66-21dd8916ef87 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:20.745797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-732c767b-fc23-4f47-8a66-21dd8916ef87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:20.759497 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c767b-fc23-4f47-8a66-21dd8916ef87 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:20.765813 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41b22c-9da6-457f-9c8c-b8e323304844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:20.766527 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41b22c-9da6-457f-9c8c-b8e323304844 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.766812 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41b22c-9da6-457f-9c8c-b8e323304844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.767134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41b22c-9da6-457f-9c8c-b8e323304844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.767902 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41b22c-9da6-457f-9c8c-b8e323304844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:20.769585 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41b22c-9da6-457f-9c8c-b8e323304844 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:20.778931 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930a567-81c0-4fb3-b02b-df4771fcef83 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:20.781917 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1431/2857] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:20.786053 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41b22c-9da6-457f-9c8c-b8e323304844 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:20.789624 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:20.790574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.791000 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.791352 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] PATH_INFO: `/v3/users/0db4d0fdf52444659936ef8fc840c4ae` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.792598 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db4d0fdf52444659936ef8fc840c4ae)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:20.794664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:20.797771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41b22c-9da6-457f-9c8c-b8e323304844 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:20.803700 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309094dc-f92c-4794-9d62-9797a4eddbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:20.804743 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309094dc-f92c-4794-9d62-9797a4eddbd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.805012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309094dc-f92c-4794-9d62-9797a4eddbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.805341 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309094dc-f92c-4794-9d62-9797a4eddbd8 admin admin] PATH_INFO: `/v3/projects/e72c59146021441f836a3b822b9c1d1e/users/dcfeaad253164dfe96a0ff1c3a578609/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.829500 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309094dc-f92c-4794-9d62-9797a4eddbd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e72c59146021441f836a3b822b9c1d1e, user_id=dcfeaad253164dfe96a0ff1c3a578609, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:20.832391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309094dc-f92c-4794-9d62-9797a4eddbd8 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:20.850171 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1432/2858] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:20 2022] PUT /identity/v3/projects/e72c59146021441f836a3b822b9c1d1e/users/dcfeaad253164dfe96a0ff1c3a578609/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:20.855121 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56104001-1293-4ebe-9faa-9777755d145b None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:20.855523 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56104001-1293-4ebe-9faa-9777755d145b None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:20.855729 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56104001-1293-4ebe-9faa-9777755d145b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:20.855952 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db4d0fdf52444659936ef8fc840c4ae'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.864937 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db4d0fdf52444659936ef8fc840c4ae'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.874619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db4d0fdf52444659936ef8fc840c4ae'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.875346 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db4d0fdf52444659936ef8fc840c4ae'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.876127 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db4d0fdf52444659936ef8fc840c4ae'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:20.879759 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-56104001-1293-4ebe-9faa-9777755d145b None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:20.881132 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-56104001-1293-4ebe-9faa-9777755d145b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:20.899763 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1427/2859] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:20 2022] DELETE /identity/v3/users/0db4d0fdf52444659936ef8fc840c4ae => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:20.937184 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:20.960632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1433/2860] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:20.971049 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:20.972616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:20.994017 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:21.050448 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bcdab-d808-4cab-9fea-0a2b2bf61028 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:21.055341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b167b-5d68-46fa-b336-942acb4390bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:21.056336 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b167b-5d68-46fa-b336-942acb4390bf service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:21.056759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b167b-5d68-46fa-b336-942acb4390bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:21.057176 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b167b-5d68-46fa-b336-942acb4390bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:21.135130 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b167b-5d68-46fa-b336-942acb4390bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:21.136857 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b167b-5d68-46fa-b336-942acb4390bf service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:21.149129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1428/2861] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:21.253791 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56104001-1293-4ebe-9faa-9777755d145b None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:21.256573 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-56104001-1293-4ebe-9faa-9777755d145b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:21.270195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56104001-1293-4ebe-9faa-9777755d145b None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:21.277204 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:21.278177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:21.278526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:21.279874 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] PATH_INFO: `/v3/projects/a34db9ee4e29452ebc4a2599a53c64eb` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:21.280323 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a34db9ee4e29452ebc4a2599a53c64eb)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:21.282916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:21.351020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a34db9ee4e29452ebc4a2599a53c64eb', 'request_id': 'req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:21.354220 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a34db9ee4e29452ebc4a2599a53c64eb', 'request_id': 'req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:21.360139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a34db9ee4e29452ebc4a2599a53c64eb', 'request_id': 'req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:21.380747 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] The token cache is being invalidate because project a34db9ee4e29452ebc4a2599a53c64eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:21.382271 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:21.383208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-6efea4c5-1bee-4b83-b24b-2c9f87aefff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:21.386783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1434/2862] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:21 2022] DELETE /identity/v3/projects/a34db9ee4e29452ebc4a2599a53c64eb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:21.392649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da10d3-dc7e-4b2d-9945-fe63bfa549d0 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:21.393637 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da10d3-dc7e-4b2d-9945-fe63bfa549d0 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:21.395708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da10d3-dc7e-4b2d-9945-fe63bfa549d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:21.422269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-13da10d3-dc7e-4b2d-9945-fe63bfa549d0 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:21.423671 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-13da10d3-dc7e-4b2d-9945-fe63bfa549d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:21.527802 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1429/2863] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:21.535887 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7d861-4f6e-45a2-993b-c8bd4f7fc71e None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:21.537287 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7d861-4f6e-45a2-993b-c8bd4f7fc71e None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:21.537555 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7d861-4f6e-45a2-993b-c8bd4f7fc71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:21.564797 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c6b7d861-4f6e-45a2-993b-c8bd4f7fc71e None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:21.566489 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c6b7d861-4f6e-45a2-993b-c8bd4f7fc71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:21.577649 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1435/2864] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:21.582353 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da10d3-dc7e-4b2d-9945-fe63bfa549d0 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:21.598530 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da10d3-dc7e-4b2d-9945-fe63bfa549d0 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:21.605128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c873e-5f65-46ee-beb1-d2390377b8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:21.605866 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c873e-5f65-46ee-beb1-d2390377b8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:21.606239 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c873e-5f65-46ee-beb1-d2390377b8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:21.606521 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c873e-5f65-46ee-beb1-d2390377b8e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:21.607285 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c873e-5f65-46ee-beb1-d2390377b8e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:30:21.607605 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c873e-5f65-46ee-beb1-d2390377b8e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:21.609381 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c873e-5f65-46ee-beb1-d2390377b8e5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:21.628200 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1430/2865] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:30:21.632442 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7d861-4f6e-45a2-993b-c8bd4f7fc71e None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:21.643482 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7d861-4f6e-45a2-993b-c8bd4f7fc71e None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:21.667006 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298fcfe-7e07-42d6-a2e2-58b48d0c45c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:21.669430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298fcfe-7e07-42d6-a2e2-58b48d0c45c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:21.669719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298fcfe-7e07-42d6-a2e2-58b48d0c45c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:21.670000 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298fcfe-7e07-42d6-a2e2-58b48d0c45c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:21.670716 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298fcfe-7e07-42d6-a2e2-58b48d0c45c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:21.673128 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298fcfe-7e07-42d6-a2e2-58b48d0c45c7 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:21.711459 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1436/2866] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:21 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:21.715018 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c873e-5f65-46ee-beb1-d2390377b8e5 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:21.731370 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c873e-5f65-46ee-beb1-d2390377b8e5 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:21.790747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:21.804066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1a927-bcdf-48b0-88a4-41aa1c9eaf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:21.805021 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1a927-bcdf-48b0-88a4-41aa1c9eaf2d admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:21.805402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1a927-bcdf-48b0-88a4-41aa1c9eaf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:21.805774 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1a927-bcdf-48b0-88a4-41aa1c9eaf2d admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:21.807143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1a927-bcdf-48b0-88a4-41aa1c9eaf2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:21.809032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1a927-bcdf-48b0-88a4-41aa1c9eaf2d admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:21.835174 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1431/2867] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:21 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:21.839636 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298fcfe-7e07-42d6-a2e2-58b48d0c45c7 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:21.851595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298fcfe-7e07-42d6-a2e2-58b48d0c45c7 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:21.869540 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534effcc-5ec1-406e-99b3-7a3fa369d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:21.870510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534effcc-5ec1-406e-99b3-7a3fa369d43b admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:21.871206 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534effcc-5ec1-406e-99b3-7a3fa369d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:21.871520 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534effcc-5ec1-406e-99b3-7a3fa369d43b admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:21.872259 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534effcc-5ec1-406e-99b3-7a3fa369d43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:21.874263 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534effcc-5ec1-406e-99b3-7a3fa369d43b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:21.890290 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1437/2868] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:21.895464 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1a927-bcdf-48b0-88a4-41aa1c9eaf2d admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:21.913751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1a927-bcdf-48b0-88a4-41aa1c9eaf2d admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:21.917834 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4a1ad-6c4d-4cb1-94c5-d4b95c7f5b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:21.918634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a1ad-6c4d-4cb1-94c5-d4b95c7f5b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:21.919029 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a1ad-6c4d-4cb1-94c5-d4b95c7f5b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:21.919485 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a1ad-6c4d-4cb1-94c5-d4b95c7f5b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:21.920300 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4a1ad-6c4d-4cb1-94c5-d4b95c7f5b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:21.921956 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4a1ad-6c4d-4cb1-94c5-d4b95c7f5b47 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:21.933826 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1432/2869] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:21.939595 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534effcc-5ec1-406e-99b3-7a3fa369d43b admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:21.948407 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534effcc-5ec1-406e-99b3-7a3fa369d43b admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:21.953801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17439b4-bdc3-40df-911d-2e2790977c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:21.954739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17439b4-bdc3-40df-911d-2e2790977c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:21.955026 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17439b4-bdc3-40df-911d-2e2790977c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:21.955332 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17439b4-bdc3-40df-911d-2e2790977c99 admin admin] PATH_INFO: `/v3/projects/b64d7fb4b6a6410084db2857c2b47091/users/e29fd364cf1243a9b790d93451258355/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:21.971706 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17439b4-bdc3-40df-911d-2e2790977c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b64d7fb4b6a6410084db2857c2b47091, user_id=e29fd364cf1243a9b790d93451258355, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:21.975021 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17439b4-bdc3-40df-911d-2e2790977c99 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:21.997670 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1438/2870] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:21 2022] PUT /identity/v3/projects/b64d7fb4b6a6410084db2857c2b47091/users/e29fd364cf1243a9b790d93451258355/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:22.009609 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce350701-cd80-4fc0-8c02-f4e7d1280bf8 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:22.010150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce350701-cd80-4fc0-8c02-f4e7d1280bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:22.010289 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce350701-cd80-4fc0-8c02-f4e7d1280bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:22.042440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ce350701-cd80-4fc0-8c02-f4e7d1280bf8 None None] MFA Rules not processed for user `e29fd364cf1243a9b790d93451258355`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:22.043905 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ce350701-cd80-4fc0-8c02-f4e7d1280bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:22.125221 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1433/2871] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:22.138437 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17439b4-bdc3-40df-911d-2e2790977c99 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:22.141578 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f17439b4-bdc3-40df-911d-2e2790977c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:22.223416 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17439b4-bdc3-40df-911d-2e2790977c99 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:22.233152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839b343-52f5-465b-8526-6fd7ee72f3bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f63e110260e4471b0a54576d3573583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:22.234092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839b343-52f5-465b-8526-6fd7ee72f3bf service cinder] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:22.234318 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839b343-52f5-465b-8526-6fd7ee72f3bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:22.234591 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839b343-52f5-465b-8526-6fd7ee72f3bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:22.250833 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839b343-52f5-465b-8526-6fd7ee72f3bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:22.253196 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839b343-52f5-465b-8526-6fd7ee72f3bf service cinder] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:22.265425 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1439/2872] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:22.293150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce350701-cd80-4fc0-8c02-f4e7d1280bf8 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:22.373674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce350701-cd80-4fc0-8c02-f4e7d1280bf8 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:22.379270 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:22.380182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:22.380454 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:22.380739 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] PATH_INFO: `/v3/users/e29fd364cf1243a9b790d93451258355` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:22.382926 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e29fd364cf1243a9b790d93451258355)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:22.384922 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:22.494953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e29fd364cf1243a9b790d93451258355'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:22.525614 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e29fd364cf1243a9b790d93451258355'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:22.537349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e29fd364cf1243a9b790d93451258355'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:22.538196 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e29fd364cf1243a9b790d93451258355'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:22.539568 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e29fd364cf1243a9b790d93451258355'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:22.541453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:22.569698 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1434/2873] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:22 2022] DELETE /identity/v3/users/e29fd364cf1243a9b790d93451258355 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:22.575623 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b442bc-dc2a-43fe-9772-9528a54b9ed8 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:22.576207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b442bc-dc2a-43fe-9772-9528a54b9ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:22.576949 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b442bc-dc2a-43fe-9772-9528a54b9ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:22.595173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:22.608631 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-c4b442bc-dc2a-43fe-9772-9528a54b9ed8 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:22.617774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c4b442bc-dc2a-43fe-9772-9528a54b9ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:22.723420 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1440/2874] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:22.734534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:22.736534 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:22.826567 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45762b10-6e64-42fd-925f-c81590c88d01 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:22.832007 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02588351-99ec-43ab-8c98-58681bf9ced9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:22.832984 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588351-99ec-43ab-8c98-58681bf9ced9 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:22.833877 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588351-99ec-43ab-8c98-58681bf9ced9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:22.834357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588351-99ec-43ab-8c98-58681bf9ced9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:22.851836 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02588351-99ec-43ab-8c98-58681bf9ced9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:22.853525 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02588351-99ec-43ab-8c98-58681bf9ced9 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:22.874449 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1435/2875] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:22.959082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b442bc-dc2a-43fe-9772-9528a54b9ed8 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:22.965458 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-c4b442bc-dc2a-43fe-9772-9528a54b9ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:22.996488 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b442bc-dc2a-43fe-9772-9528a54b9ed8 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:23.005368 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:23.006439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:23.006771 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:23.007050 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] PATH_INFO: `/v3/users/c1785d0b419643aa90fd3e9146a5c6c6` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:23.012502 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1785d0b419643aa90fd3e9146a5c6c6)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:23.015901 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:23.104400 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1785d0b419643aa90fd3e9146a5c6c6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:23.113969 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1785d0b419643aa90fd3e9146a5c6c6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:23.115119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1785d0b419643aa90fd3e9146a5c6c6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:23.128381 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1785d0b419643aa90fd3e9146a5c6c6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:23.146233 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1785d0b419643aa90fd3e9146a5c6c6'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:23.155572 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1441/2876] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:22 2022] DELETE /identity/v3/users/c1785d0b419643aa90fd3e9146a5c6c6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:23.160493 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d3cc2-f50f-4cd5-b50f-10f109919d2d None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:23.162476 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d3cc2-f50f-4cd5-b50f-10f109919d2d None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:23.163033 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d3cc2-f50f-4cd5-b50f-10f109919d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:23.189159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-845d3cc2-f50f-4cd5-b50f-10f109919d2d None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:23.191661 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-845d3cc2-f50f-4cd5-b50f-10f109919d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:23.297470 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1436/2877] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:23.306707 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:23.308203 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:23.346872 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d3cc2-f50f-4cd5-b50f-10f109919d2d None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:23.348510 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-845d3cc2-f50f-4cd5-b50f-10f109919d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:23.366727 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d3cc2-f50f-4cd5-b50f-10f109919d2d None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:23.377754 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:23.379307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:23.379766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:23.380215 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] PATH_INFO: `/v3/projects/b64d7fb4b6a6410084db2857c2b47091` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:23.382312 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b64d7fb4b6a6410084db2857c2b47091)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:23.385189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:23.426045 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc700c-b539-411c-adce-67bfd95f2402 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:23.430316 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001adf46-f0ed-4f8e-802f-26636f4ac059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:23.430983 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001adf46-f0ed-4f8e-802f-26636f4ac059 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:23.431324 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001adf46-f0ed-4f8e-802f-26636f4ac059 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:23.432901 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001adf46-f0ed-4f8e-802f-26636f4ac059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:23.443474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001adf46-f0ed-4f8e-802f-26636f4ac059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:23.445064 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001adf46-f0ed-4f8e-802f-26636f4ac059 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:23.456316 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b64d7fb4b6a6410084db2857c2b47091', 'request_id': 'req-31316726-0096-4066-a6fe-03b1b2cd1aa7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:23.459509 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1442/2878] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:23.482153 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b64d7fb4b6a6410084db2857c2b47091', 'request_id': 'req-31316726-0096-4066-a6fe-03b1b2cd1aa7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:23.494856 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b64d7fb4b6a6410084db2857c2b47091', 'request_id': 'req-31316726-0096-4066-a6fe-03b1b2cd1aa7'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:23.528365 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] The token cache is being invalidate because project b64d7fb4b6a6410084db2857c2b47091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:23.528665 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:23.531021 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-31316726-0096-4066-a6fe-03b1b2cd1aa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:23.535307 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1437/2879] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:23 2022] DELETE /identity/v3/projects/b64d7fb4b6a6410084db2857c2b47091 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:23.928495 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001adf46-f0ed-4f8e-802f-26636f4ac059 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:23.929974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-001adf46-f0ed-4f8e-802f-26636f4ac059 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:23.945957 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001adf46-f0ed-4f8e-802f-26636f4ac059 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:23.993153 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:24.044456 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:24.045402 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.045764 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.046091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] PATH_INFO: `/v3/projects/8cfe127799e0444d959228abe7ffe5e1` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.047364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cfe127799e0444d959228abe7ffe5e1)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:24.050842 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:24.109691 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cfe127799e0444d959228abe7ffe5e1', 'request_id': 'req-f1aab735-60bb-4372-899b-e4814efe6367'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:24.117762 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cfe127799e0444d959228abe7ffe5e1', 'request_id': 'req-f1aab735-60bb-4372-899b-e4814efe6367'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:24.124490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cfe127799e0444d959228abe7ffe5e1', 'request_id': 'req-f1aab735-60bb-4372-899b-e4814efe6367'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:24.148569 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] The token cache is being invalidate because project 8cfe127799e0444d959228abe7ffe5e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80128) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:24.148924 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:24.149664 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-f1aab735-60bb-4372-899b-e4814efe6367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:24.152490 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1443/2880] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:23 2022] DELETE /identity/v3/projects/8cfe127799e0444d959228abe7ffe5e1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:24.157647 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e359-24f6-40d4-a6b0-2322c5c5de8c None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.158138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e359-24f6-40d4-a6b0-2322c5c5de8c None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.158666 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e359-24f6-40d4-a6b0-2322c5c5de8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.179500 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:24.186115 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c548e359-24f6-40d4-a6b0-2322c5c5de8c None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:24.188320 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c548e359-24f6-40d4-a6b0-2322c5c5de8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:24.261142 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1438/2881] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:24.269653 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d08ae0-cdb8-46fa-a680-79634563b518 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.270166 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d08ae0-cdb8-46fa-a680-79634563b518 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.270584 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d08ae0-cdb8-46fa-a680-79634563b518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.301354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-14d08ae0-cdb8-46fa-a680-79634563b518 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:24.302582 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-14d08ae0-cdb8-46fa-a680-79634563b518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:24.311918 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1444/2882] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:24.316030 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e359-24f6-40d4-a6b0-2322c5c5de8c None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:24.330206 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e359-24f6-40d4-a6b0-2322c5c5de8c None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:24.337753 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd280ea-9fd2-49e1-a477-746bf1142046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:24.338843 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd280ea-9fd2-49e1-a477-746bf1142046 admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.339455 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd280ea-9fd2-49e1-a477-746bf1142046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.339889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd280ea-9fd2-49e1-a477-746bf1142046 admin admin] PATH_INFO: `/v3/domains` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.340766 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd280ea-9fd2-49e1-a477-746bf1142046 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80129) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 06:30:24.341209 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd280ea-9fd2-49e1-a477-746bf1142046 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:24.347488 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd280ea-9fd2-49e1-a477-746bf1142046 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:24.364783 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1439/2883] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jul 4 06:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 06:30:24.369020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d08ae0-cdb8-46fa-a680-79634563b518 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:24.379894 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d08ae0-cdb8-46fa-a680-79634563b518 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:24.385893 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affc4ea-d898-4eda-82f0-bc777150fa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:24.386581 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affc4ea-d898-4eda-82f0-bc777150fa43 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.386864 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affc4ea-d898-4eda-82f0-bc777150fa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.387193 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affc4ea-d898-4eda-82f0-bc777150fa43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.387961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affc4ea-d898-4eda-82f0-bc777150fa43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:24.389973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affc4ea-d898-4eda-82f0-bc777150fa43 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:24.430371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1445/2884] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:24 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:24.435751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd280ea-9fd2-49e1-a477-746bf1142046 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:24.447139 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd280ea-9fd2-49e1-a477-746bf1142046 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:24.538181 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9731d-b035-4951-9a10-af96255761fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:24.538882 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9731d-b035-4951-9a10-af96255761fc admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.539499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9731d-b035-4951-9a10-af96255761fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.539499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9731d-b035-4951-9a10-af96255761fc admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.540189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9731d-b035-4951-9a10-af96255761fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:24.542705 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9731d-b035-4951-9a10-af96255761fc admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:24.572596 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1440/2885] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:24 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:24.576611 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affc4ea-d898-4eda-82f0-bc777150fa43 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:24.587576 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affc4ea-d898-4eda-82f0-bc777150fa43 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:24.593199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cf5b7-92b0-4c08-9254-24761eb33356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:24.593739 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cf5b7-92b0-4c08-9254-24761eb33356 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.594020 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cf5b7-92b0-4c08-9254-24761eb33356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.594292 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cf5b7-92b0-4c08-9254-24761eb33356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.595278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cf5b7-92b0-4c08-9254-24761eb33356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:24.596906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cf5b7-92b0-4c08-9254-24761eb33356 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:24.611326 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1446/2886] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:24.612717 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9731d-b035-4951-9a10-af96255761fc admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:24.628190 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9731d-b035-4951-9a10-af96255761fc admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:24.633818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c6db6-9042-4da1-b924-25edd05b546b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:24.634522 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c6db6-9042-4da1-b924-25edd05b546b admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.634798 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c6db6-9042-4da1-b924-25edd05b546b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.635110 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c6db6-9042-4da1-b924-25edd05b546b admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.635933 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c6db6-9042-4da1-b924-25edd05b546b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:24.637546 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c6db6-9042-4da1-b924-25edd05b546b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:24.655001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1441/2887] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:24.656392 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cf5b7-92b0-4c08-9254-24761eb33356 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:24.673625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cf5b7-92b0-4c08-9254-24761eb33356 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:24.680361 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fdfc8-bf8b-4280-bd77-c28655316566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:24.681364 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fdfc8-bf8b-4280-bd77-c28655316566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.682177 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fdfc8-bf8b-4280-bd77-c28655316566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.682466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fdfc8-bf8b-4280-bd77-c28655316566 admin admin] PATH_INFO: `/v3/projects/764c6dcba64244dba2aff169c181ccea/users/82e1f29c9cda427cac832f4481fd0b26/roles/d0a32856510a480a897677657ace20de` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.701997 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fdfc8-bf8b-4280-bd77-c28655316566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=764c6dcba64244dba2aff169c181ccea, user_id=82e1f29c9cda427cac832f4481fd0b26, role_id=d0a32856510a480a897677657ace20de)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:24.706108 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fdfc8-bf8b-4280-bd77-c28655316566 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:24.728192 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1447/2888] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:24 2022] PUT /identity/v3/projects/764c6dcba64244dba2aff169c181ccea/users/82e1f29c9cda427cac832f4481fd0b26/roles/d0a32856510a480a897677657ace20de => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:24.733631 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf39fe5-a524-460a-88e1-9980774cd888 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.733919 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf39fe5-a524-460a-88e1-9980774cd888 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.734194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf39fe5-a524-460a-88e1-9980774cd888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.753343 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-aaf39fe5-a524-460a-88e1-9980774cd888 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:24.754241 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-aaf39fe5-a524-460a-88e1-9980774cd888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:24.840187 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1442/2889] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:24.851231 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fdfc8-bf8b-4280-bd77-c28655316566 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:24.853031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d77fdfc8-bf8b-4280-bd77-c28655316566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:24.935832 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:24.979228 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fdfc8-bf8b-4280-bd77-c28655316566 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:24.983624 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ce224-e275-46e3-9b4e-3af079a6c160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:24.984464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ce224-e275-46e3-9b4e-3af079a6c160 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:24.984964 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ce224-e275-46e3-9b4e-3af079a6c160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:24.985399 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ce224-e275-46e3-9b4e-3af079a6c160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:24.998866 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ce224-e275-46e3-9b4e-3af079a6c160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:25.000863 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ce224-e275-46e3-9b4e-3af079a6c160 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:25.041242 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1448/2890] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:26.292960 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600408d-b835-4d00-bd4f-e896a785fa3a None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:26.293818 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600408d-b835-4d00-bd4f-e896a785fa3a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:26.294245 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600408d-b835-4d00-bd4f-e896a785fa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:26.327916 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:26.335869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a600408d-b835-4d00-bd4f-e896a785fa3a None None] MFA Rules not processed for user `dcfeaad253164dfe96a0ff1c3a578609`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:26.337189 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a600408d-b835-4d00-bd4f-e896a785fa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:26.436322 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1443/2891] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:26.440526 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ce224-e275-46e3-9b4e-3af079a6c160 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:26.443524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-a63ce224-e275-46e3-9b4e-3af079a6c160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:26.460255 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ce224-e275-46e3-9b4e-3af079a6c160 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:26.467958 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850434f-78eb-43b5-9e7e-649a371e7a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:26.469586 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850434f-78eb-43b5-9e7e-649a371e7a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:26.469922 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850434f-78eb-43b5-9e7e-649a371e7a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:26.470276 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850434f-78eb-43b5-9e7e-649a371e7a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:26.471036 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850434f-78eb-43b5-9e7e-649a371e7a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:26.473514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850434f-78eb-43b5-9e7e-649a371e7a74 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:26.508794 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1449/2892] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:26 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 06:30:26.512496 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600408d-b835-4d00-bd4f-e896a785fa3a None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:26.528619 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600408d-b835-4d00-bd4f-e896a785fa3a None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:26.631086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9074a3-b59f-491b-859f-bd0eaccf101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:26.631374 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9074a3-b59f-491b-859f-bd0eaccf101b admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:26.631578 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9074a3-b59f-491b-859f-bd0eaccf101b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:26.631842 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9074a3-b59f-491b-859f-bd0eaccf101b admin admin] PATH_INFO: `/v3/users` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:26.632584 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9074a3-b59f-491b-859f-bd0eaccf101b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:26.638161 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9074a3-b59f-491b-859f-bd0eaccf101b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:26.668452 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1444/2893] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:26 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:26.672387 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850434f-78eb-43b5-9e7e-649a371e7a74 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:26.683422 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850434f-78eb-43b5-9e7e-649a371e7a74 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:26.689173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7461d-f1d8-444d-b042-730a71052687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:26.689173 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7461d-f1d8-444d-b042-730a71052687 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:26.689429 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7461d-f1d8-444d-b042-730a71052687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:26.689538 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7461d-f1d8-444d-b042-730a71052687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:26.690288 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7461d-f1d8-444d-b042-730a71052687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:26.692134 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7461d-f1d8-444d-b042-730a71052687 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:26.705982 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1450/2894] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 06:30:26.713556 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9074a3-b59f-491b-859f-bd0eaccf101b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:26.730737 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9074a3-b59f-491b-859f-bd0eaccf101b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:26.736900 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a0d50-a9e0-4d96-aeaf-1cd3d8381f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:26.737886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0d50-a9e0-4d96-aeaf-1cd3d8381f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:26.738232 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0d50-a9e0-4d96-aeaf-1cd3d8381f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:26.738566 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0d50-a9e0-4d96-aeaf-1cd3d8381f46 admin admin] PATH_INFO: `/v3/projects/7cec5546b49e4dde8dbe6c44947790e2/users/c936b59e3c6f48478c30de3d3dd7a34b/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:26.757672 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a0d50-a9e0-4d96-aeaf-1cd3d8381f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cec5546b49e4dde8dbe6c44947790e2, user_id=c936b59e3c6f48478c30de3d3dd7a34b, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:26.760423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a0d50-a9e0-4d96-aeaf-1cd3d8381f46 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:26.776649 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1445/2895] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:26 2022] PUT /identity/v3/projects/7cec5546b49e4dde8dbe6c44947790e2/users/c936b59e3c6f48478c30de3d3dd7a34b/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:26.782257 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7461d-f1d8-444d-b042-730a71052687 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:26.801194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7461d-f1d8-444d-b042-730a71052687 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:26.900445 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ece68-3114-43bb-a553-42fb13e72755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:26.901371 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ece68-3114-43bb-a553-42fb13e72755 admin admin] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:26.901942 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ece68-3114-43bb-a553-42fb13e72755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:26.902857 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ece68-3114-43bb-a553-42fb13e72755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:26.903906 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ece68-3114-43bb-a553-42fb13e72755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:26.905719 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ece68-3114-43bb-a553-42fb13e72755 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:26.923367 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1451/2896] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:26.924776 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a0d50-a9e0-4d96-aeaf-1cd3d8381f46 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:26.941417 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a0d50-a9e0-4d96-aeaf-1cd3d8381f46 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:26.946878 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111737f-e0ce-4fa3-9b96-8981cc051d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:26.947891 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111737f-e0ce-4fa3-9b96-8981cc051d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:26.948129 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111737f-e0ce-4fa3-9b96-8981cc051d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:26.948407 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111737f-e0ce-4fa3-9b96-8981cc051d37 admin admin] PATH_INFO: `/v3/projects/7cec5546b49e4dde8dbe6c44947790e2/users/c936b59e3c6f48478c30de3d3dd7a34b/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:26.958308 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111737f-e0ce-4fa3-9b96-8981cc051d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cec5546b49e4dde8dbe6c44947790e2, user_id=c936b59e3c6f48478c30de3d3dd7a34b, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:26.961205 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111737f-e0ce-4fa3-9b96-8981cc051d37 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:26.975303 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1446/2897] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:26 2022] PUT /identity/v3/projects/7cec5546b49e4dde8dbe6c44947790e2/users/c936b59e3c6f48478c30de3d3dd7a34b/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 06:30:31.067965 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c213d8b-720a-433f-9d67-f7e316a69c11 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:31.068474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c213d8b-720a-433f-9d67-f7e316a69c11 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:31.068552 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c213d8b-720a-433f-9d67-f7e316a69c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:31.143612 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-6c213d8b-720a-433f-9d67-f7e316a69c11 None None] MFA Rules not processed for user `82e1f29c9cda427cac832f4481fd0b26`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:31.145533 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-6c213d8b-720a-433f-9d67-f7e316a69c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:31.260605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1452/2898] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:31.265229 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111737f-e0ce-4fa3-9b96-8981cc051d37 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:31.430138 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:31.493159 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111737f-e0ce-4fa3-9b96-8981cc051d37 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:31.520043 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5aff7-c325-4a9c-bfec-c2d76b873ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:31.520747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5aff7-c325-4a9c-bfec-c2d76b873ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:31.521031 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5aff7-c325-4a9c-bfec-c2d76b873ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:31.521304 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5aff7-c325-4a9c-bfec-c2d76b873ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:31.522195 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5aff7-c325-4a9c-bfec-c2d76b873ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:31.523983 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5aff7-c325-4a9c-bfec-c2d76b873ad6 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:31.627574 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1447/2899] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jul 4 06:30:31 2022] POST /identity/v3/projects => generated 385 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:31.631618 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c213d8b-720a-433f-9d67-f7e316a69c11 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:31.648947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c213d8b-720a-433f-9d67-f7e316a69c11 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:31.701091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:31.777736 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18c809-1596-4076-970b-0f5db26098d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:31.778472 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18c809-1596-4076-970b-0f5db26098d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:31.778873 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18c809-1596-4076-970b-0f5db26098d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:31.779217 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18c809-1596-4076-970b-0f5db26098d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:31.780726 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18c809-1596-4076-970b-0f5db26098d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:31.782765 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18c809-1596-4076-970b-0f5db26098d5 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:31.814012 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1453/2900] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jul 4 06:30:31 2022] POST /identity/v3/users => generated 299 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 06:30:31.817747 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5aff7-c325-4a9c-bfec-c2d76b873ad6 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:31.833936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5aff7-c325-4a9c-bfec-c2d76b873ad6 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:31.842781 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929ea60-c3b9-4ce1-9c43-2b2a6b0eb09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:31.843814 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929ea60-c3b9-4ce1-9c43-2b2a6b0eb09f admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:31.844258 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929ea60-c3b9-4ce1-9c43-2b2a6b0eb09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:31.844749 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929ea60-c3b9-4ce1-9c43-2b2a6b0eb09f admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:31.845650 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929ea60-c3b9-4ce1-9c43-2b2a6b0eb09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:31.853236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929ea60-c3b9-4ce1-9c43-2b2a6b0eb09f admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:31.867460 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1448/2901] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:31.869668 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18c809-1596-4076-970b-0f5db26098d5 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:31.881868 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18c809-1596-4076-970b-0f5db26098d5 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:31.887159 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848475c-216b-4c74-9a15-ddec8a9a7d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:31.888265 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848475c-216b-4c74-9a15-ddec8a9a7d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:31.888652 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848475c-216b-4c74-9a15-ddec8a9a7d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:31.889091 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848475c-216b-4c74-9a15-ddec8a9a7d68 admin admin] PATH_INFO: `/v3/projects/ca6c7523fe114d4ab10abade04851059/users/8f25016c9ef74db18317f03bf018d409/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:31.906256 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848475c-216b-4c74-9a15-ddec8a9a7d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6c7523fe114d4ab10abade04851059, user_id=8f25016c9ef74db18317f03bf018d409, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:31.909496 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848475c-216b-4c74-9a15-ddec8a9a7d68 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:31.943089 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1454/2902] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:31 2022] PUT /identity/v3/projects/ca6c7523fe114d4ab10abade04851059/users/8f25016c9ef74db18317f03bf018d409/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:31.944418 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929ea60-c3b9-4ce1-9c43-2b2a6b0eb09f admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:31.955846 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929ea60-c3b9-4ce1-9c43-2b2a6b0eb09f admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:32.029357 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147fc5b-453b-4a2b-b27c-2e177c1d6f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:32.030880 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147fc5b-453b-4a2b-b27c-2e177c1d6f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:32.032768 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147fc5b-453b-4a2b-b27c-2e177c1d6f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:32.033395 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147fc5b-453b-4a2b-b27c-2e177c1d6f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:32.034830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147fc5b-453b-4a2b-b27c-2e177c1d6f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:32.036884 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147fc5b-453b-4a2b-b27c-2e177c1d6f7c admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:32.053128 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1449/2903] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jul 4 06:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 06:30:32.057385 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848475c-216b-4c74-9a15-ddec8a9a7d68 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:32.087821 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848475c-216b-4c74-9a15-ddec8a9a7d68 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:32.093505 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237ea99-3b70-4dfa-b608-ebc398a6fe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:32.094439 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237ea99-3b70-4dfa-b608-ebc398a6fe4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:32.094755 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237ea99-3b70-4dfa-b608-ebc398a6fe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:32.095031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237ea99-3b70-4dfa-b608-ebc398a6fe4b admin admin] PATH_INFO: `/v3/projects/ca6c7523fe114d4ab10abade04851059/users/8f25016c9ef74db18317f03bf018d409/roles/8d5c4a5f0b2a44c2934090dd98122f95` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:32.102137 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237ea99-3b70-4dfa-b608-ebc398a6fe4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6c7523fe114d4ab10abade04851059, user_id=8f25016c9ef74db18317f03bf018d409, role_id=8d5c4a5f0b2a44c2934090dd98122f95)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:32.119264 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237ea99-3b70-4dfa-b608-ebc398a6fe4b admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:32.136801 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1455/2904] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jul 4 06:30:32 2022] PUT /identity/v3/projects/ca6c7523fe114d4ab10abade04851059/users/8f25016c9ef74db18317f03bf018d409/roles/8d5c4a5f0b2a44c2934090dd98122f95 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:33.131309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a63c8-f8ed-4076-aa2a-459ba3bead7f None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:33.131830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a63c8-f8ed-4076-aa2a-459ba3bead7f None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:33.131830 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a63c8-f8ed-4076-aa2a-459ba3bead7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:33.174883 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a06a63c8-f8ed-4076-aa2a-459ba3bead7f None None] MFA Rules not processed for user `c936b59e3c6f48478c30de3d3dd7a34b`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:33.177172 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a06a63c8-f8ed-4076-aa2a-459ba3bead7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:33.319788 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1450/2905] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:33.362780 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237ea99-3b70-4dfa-b608-ebc398a6fe4b admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:33.364810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7237ea99-3b70-4dfa-b608-ebc398a6fe4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:33.614512 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237ea99-3b70-4dfa-b608-ebc398a6fe4b admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:33.625084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25bf8e-5b4a-4398-bcb4-37a327493ca5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:33.625980 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25bf8e-5b4a-4398-bcb4-37a327493ca5 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:33.626263 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25bf8e-5b4a-4398-bcb4-37a327493ca5 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:33.626859 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25bf8e-5b4a-4398-bcb4-37a327493ca5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:33.781644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25bf8e-5b4a-4398-bcb4-37a327493ca5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:33.783558 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25bf8e-5b4a-4398-bcb4-37a327493ca5 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:33.808278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1456/2906] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:36.017397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb726f-8252-437d-bcce-a3099867b6ef None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:36.018091 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb726f-8252-437d-bcce-a3099867b6ef None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:36.018091 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb726f-8252-437d-bcce-a3099867b6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:36.058137 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-14cb726f-8252-437d-bcce-a3099867b6ef None None] MFA Rules not processed for user `dcfeaad253164dfe96a0ff1c3a578609`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:36.059462 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-14cb726f-8252-437d-bcce-a3099867b6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:36.092907 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1451/2907] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:39.776734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d886c7f-09c9-4776-95b2-93d437b20a09 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:39.777869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d886c7f-09c9-4776-95b2-93d437b20a09 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:39.778226 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d886c7f-09c9-4776-95b2-93d437b20a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:39.805474 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:39.810349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-7d886c7f-09c9-4776-95b2-93d437b20a09 None None] MFA Rules not processed for user `8f25016c9ef74db18317f03bf018d409`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:39.813082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7d886c7f-09c9-4776-95b2-93d437b20a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:39.915883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1457/2908] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:39.932016 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb726f-8252-437d-bcce-a3099867b6ef None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:39.933778 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-14cb726f-8252-437d-bcce-a3099867b6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:39.968175 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:40.039591 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb726f-8252-437d-bcce-a3099867b6ef None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:40.043908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f72191-321f-42a5-a0f5-ec9a10d90d5d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab33473d40094ec681e538cde250ec2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:40.044828 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f72191-321f-42a5-a0f5-ec9a10d90d5d service glance] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:40.045273 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f72191-321f-42a5-a0f5-ec9a10d90d5d service glance] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:40.045702 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f72191-321f-42a5-a0f5-ec9a10d90d5d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:40.126953 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f72191-321f-42a5-a0f5-ec9a10d90d5d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:40.128927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f72191-321f-42a5-a0f5-ec9a10d90d5d service glance] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:40.142519 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1452/2909] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:40.269891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d886c7f-09c9-4776-95b2-93d437b20a09 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:40.271536 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7d886c7f-09c9-4776-95b2-93d437b20a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:40.288751 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf572b07-4a54-4a66-b80e-b56772b41ec9 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:40.289908 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf572b07-4a54-4a66-b80e-b56772b41ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:40.290204 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf572b07-4a54-4a66-b80e-b56772b41ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:40.329382 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-cf572b07-4a54-4a66-b80e-b56772b41ec9 None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:40.334724 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cf572b07-4a54-4a66-b80e-b56772b41ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:40.380421 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d886c7f-09c9-4776-95b2-93d437b20a09 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:40.388802 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:40.389707 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:40.389985 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:40.390974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] PATH_INFO: `/v3/users/1a1e8da20a9144fc8de9e40e77ef1f51` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:40.392481 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1e8da20a9144fc8de9e40e77ef1f51)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:40.394465 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:40.406936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1453/2910] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 06:30:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 06:30:40.421461 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf572b07-4a54-4a66-b80e-b56772b41ec9 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:40.424067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-cf572b07-4a54-4a66-b80e-b56772b41ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:40.484463 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1e8da20a9144fc8de9e40e77ef1f51'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:40.494885 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1e8da20a9144fc8de9e40e77ef1f51'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:40.495619 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1e8da20a9144fc8de9e40e77ef1f51'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:40.514194 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1e8da20a9144fc8de9e40e77ef1f51'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:40.541424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-518dbb50-8b16-4e62-84a1-b33bf8dc340e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1e8da20a9144fc8de9e40e77ef1f51'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:40.559734 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1458/2911] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:30:40 2022] DELETE /identity/v3/users/1a1e8da20a9144fc8de9e40e77ef1f51 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:40.565390 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f8bf3-727f-4f6c-85d8-9f8cb8d9b051 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:40.571776 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f8bf3-727f-4f6c-85d8-9f8cb8d9b051 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:40.572195 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f8bf3-727f-4f6c-85d8-9f8cb8d9b051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:40.575978 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf572b07-4a54-4a66-b80e-b56772b41ec9 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:40.609674 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=100 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:40.624660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-d23f8bf3-727f-4f6c-85d8-9f8cb8d9b051 None None] MFA Rules not processed for user `825e62d400e54cc0868912432c257354`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:40.625999 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d23f8bf3-727f-4f6c-85d8-9f8cb8d9b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:40.651593 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:40.722945 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2829905-5ea7-48a6-b7a3-f5e9117a026c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:40.724143 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2829905-5ea7-48a6-b7a3-f5e9117a026c service swift] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:40.724664 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2829905-5ea7-48a6-b7a3-f5e9117a026c service swift] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:40.725134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2829905-5ea7-48a6-b7a3-f5e9117a026c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:40.727267 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c2829905-5ea7-48a6-b7a3-f5e9117a026c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:40.763034 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1459/2912] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:40.778976 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f8bf3-727f-4f6c-85d8-9f8cb8d9b051 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:40.779961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d23f8bf3-727f-4f6c-85d8-9f8cb8d9b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:40.854506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2829905-5ea7-48a6-b7a3-f5e9117a026c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:40.856623 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2829905-5ea7-48a6-b7a3-f5e9117a026c service swift] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:40.869163 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1454/2913] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 06:30:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 06:30:40.915689 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f8bf3-727f-4f6c-85d8-9f8cb8d9b051 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:40.925634 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996636a-3333-4b75-8503-91cfd2838376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:40.926322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996636a-3333-4b75-8503-91cfd2838376 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:40.926599 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996636a-3333-4b75-8503-91cfd2838376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:40.927230 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996636a-3333-4b75-8503-91cfd2838376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:40.951278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996636a-3333-4b75-8503-91cfd2838376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:40.951278 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996636a-3333-4b75-8503-91cfd2838376 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:40.977767 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1460/2914] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:41.657183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2829905-5ea7-48a6-b7a3-f5e9117a026c service swift] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:41.657183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c2829905-5ea7-48a6-b7a3-f5e9117a026c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:41.694515 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2829905-5ea7-48a6-b7a3-f5e9117a026c service swift] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:41.719586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:41.720499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:41.720784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:41.721076 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] PATH_INFO: `/v3/projects/e16ce9238fc64bfe914286f4e429fa4c` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:41.730288 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e16ce9238fc64bfe914286f4e429fa4c)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:41.732321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:41.866182 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e16ce9238fc64bfe914286f4e429fa4c', 'request_id': 'req-628f0976-12f7-4ff1-9f6b-b211e896898a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:41.874443 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e16ce9238fc64bfe914286f4e429fa4c', 'request_id': 'req-628f0976-12f7-4ff1-9f6b-b211e896898a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:41.882888 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e16ce9238fc64bfe914286f4e429fa4c', 'request_id': 'req-628f0976-12f7-4ff1-9f6b-b211e896898a'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:41.916328 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] The token cache is being invalidate because project e16ce9238fc64bfe914286f4e429fa4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:30:41.917441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:41.918338 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-628f0976-12f7-4ff1-9f6b-b211e896898a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:30:41.931626 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1455/2915] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:30:41 2022] DELETE /identity/v3/projects/e16ce9238fc64bfe914286f4e429fa4c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:30:50.955452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=45 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:51.906118 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:30:51.909287 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996636a-3333-4b75-8503-91cfd2838376 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:51.910795 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-8996636a-3333-4b75-8503-91cfd2838376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:51.923133 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996636a-3333-4b75-8503-91cfd2838376 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:51.980494 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af43d9c-df4c-4777-b9c1-84b5d6357506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:51.981047 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af43d9c-df4c-4777-b9c1-84b5d6357506 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:51.981301 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af43d9c-df4c-4777-b9c1-84b5d6357506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:51.981570 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af43d9c-df4c-4777-b9c1-84b5d6357506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:51.983565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2af43d9c-df4c-4777-b9c1-84b5d6357506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:51.993340 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af43d9c-df4c-4777-b9c1-84b5d6357506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:51.994860 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af43d9c-df4c-4777-b9c1-84b5d6357506 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:52.004356 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1461/2916] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:52.132677 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ce9fd-df6e-459b-a945-ea30284e7f12 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:52.132995 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ce9fd-df6e-459b-a945-ea30284e7f12 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:52.133269 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ce9fd-df6e-459b-a945-ea30284e7f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:52.151758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-c99ce9fd-df6e-459b-a945-ea30284e7f12 None None] MFA Rules not processed for user `82e1f29c9cda427cac832f4481fd0b26`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:52.152301 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-c99ce9fd-df6e-459b-a945-ea30284e7f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:52.212276 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1456/2917] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jul 4 06:30:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:52.468869 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af43d9c-df4c-4777-b9c1-84b5d6357506 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:52.470362 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2af43d9c-df4c-4777-b9c1-84b5d6357506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:52.480803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af43d9c-df4c-4777-b9c1-84b5d6357506 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:52.484862 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac22e74-e4b4-49a4-b750-b130052c0512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:52.485615 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac22e74-e4b4-49a4-b750-b130052c0512 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:52.485961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac22e74-e4b4-49a4-b750-b130052c0512 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:52.486279 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac22e74-e4b4-49a4-b750-b130052c0512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:52.488204 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9ac22e74-e4b4-49a4-b750-b130052c0512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:52.503659 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac22e74-e4b4-49a4-b750-b130052c0512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:52.505152 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac22e74-e4b4-49a4-b750-b130052c0512 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:52.518464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1462/2918] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:56.156349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0436cb2-4abc-4d28-82cf-ea67ee748f0a None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:56.157634 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0436cb2-4abc-4d28-82cf-ea67ee748f0a None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:56.158071 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0436cb2-4abc-4d28-82cf-ea67ee748f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:56.182330 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-a0436cb2-4abc-4d28-82cf-ea67ee748f0a None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:30:56.184309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a0436cb2-4abc-4d28-82cf-ea67ee748f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:56.258032 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1457/2919] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 4 06:30:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:30:56.274207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac22e74-e4b4-49a4-b750-b130052c0512 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:56.276810 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9ac22e74-e4b4-49a4-b750-b130052c0512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:56.398377 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac22e74-e4b4-49a4-b750-b130052c0512 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:56.411805 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830cf51-68aa-44f3-9717-286d0cc6015e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:56.412491 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830cf51-68aa-44f3-9717-286d0cc6015e service swift] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:56.412774 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830cf51-68aa-44f3-9717-286d0cc6015e service swift] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:56.413092 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830cf51-68aa-44f3-9717-286d0cc6015e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:56.422370 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830cf51-68aa-44f3-9717-286d0cc6015e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:56.424199 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830cf51-68aa-44f3-9717-286d0cc6015e service swift] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:56.429722 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0436cb2-4abc-4d28-82cf-ea67ee748f0a None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:56.442859 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0436cb2-4abc-4d28-82cf-ea67ee748f0a None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:56.446427 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1463/2920] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 06:30:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 06:30:56.449847 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b7197-b54b-4c0f-bd46-0dfa2e658cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:56.450257 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b7197-b54b-4c0f-bd46-0dfa2e658cac service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:56.451299 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b7197-b54b-4c0f-bd46-0dfa2e658cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:56.451784 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b7197-b54b-4c0f-bd46-0dfa2e658cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:56.454281 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e36b7197-b54b-4c0f-bd46-0dfa2e658cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:56.465889 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b7197-b54b-4c0f-bd46-0dfa2e658cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:56.467453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b7197-b54b-4c0f-bd46-0dfa2e658cac service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:56.491928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1458/2921] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:30:58.406784 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830cf51-68aa-44f3-9717-286d0cc6015e service swift] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:30:58.408372 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7830cf51-68aa-44f3-9717-286d0cc6015e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:58.618973 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830cf51-68aa-44f3-9717-286d0cc6015e service swift] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:30:58.626878 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13eca35-2d06-4687-ba17-eae7b44a041c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:30:58.628476 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eca35-2d06-4687-ba17-eae7b44a041c service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:30:58.628926 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eca35-2d06-4687-ba17-eae7b44a041c service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:30:58.629349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eca35-2d06-4687-ba17-eae7b44a041c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:30:58.631498 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f13eca35-2d06-4687-ba17-eae7b44a041c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:30:58.663686 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13eca35-2d06-4687-ba17-eae7b44a041c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:30:58.665432 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13eca35-2d06-4687-ba17-eae7b44a041c service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:30:58.677916 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1464/2922] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:30:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:31:06.473506 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=56 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:31:08.672804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=98 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:31:21.412871 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b7197-b54b-4c0f-bd46-0dfa2e658cac service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:31:21.414436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-e36b7197-b54b-4c0f-bd46-0dfa2e658cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:31:21.425344 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b7197-b54b-4c0f-bd46-0dfa2e658cac service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:31:21.429321 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c996391-c5cc-41bb-9700-6f21ef2910b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:31:21.429965 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c996391-c5cc-41bb-9700-6f21ef2910b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:31:21.430244 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c996391-c5cc-41bb-9700-6f21ef2910b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:31:21.430511 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c996391-c5cc-41bb-9700-6f21ef2910b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:31:21.432408 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4c996391-c5cc-41bb-9700-6f21ef2910b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:31:21.500341 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c996391-c5cc-41bb-9700-6f21ef2910b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:31:21.501823 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c996391-c5cc-41bb-9700-6f21ef2910b2 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:31:21.514113 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1459/2923] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:31:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 06:31:24.680478 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13eca35-2d06-4687-ba17-eae7b44a041c service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:31:24.695514 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13eca35-2d06-4687-ba17-eae7b44a041c service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:31:24.700139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc17ce4-3560-4878-abdb-a7851ba669a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:31:24.700783 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc17ce4-3560-4878-abdb-a7851ba669a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:31:24.701066 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc17ce4-3560-4878-abdb-a7851ba669a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:31:24.701322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc17ce4-3560-4878-abdb-a7851ba669a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:31:24.703565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-2cc17ce4-3560-4878-abdb-a7851ba669a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:31:24.768461 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc17ce4-3560-4878-abdb-a7851ba669a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:31:24.769914 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc17ce4-3560-4878-abdb-a7851ba669a0 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:31:24.779879 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1465/2924] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:31:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:31:31.508302 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=30 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:31:34.775565 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=30 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:32:35.038046 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c996391-c5cc-41bb-9700-6f21ef2910b2 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:32:35.040397 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-4c996391-c5cc-41bb-9700-6f21ef2910b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:32:35.128280 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c996391-c5cc-41bb-9700-6f21ef2910b2 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:32:35.133659 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff59ff-b5af-4d73-a488-22f2e9fdacf0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:32:35.134419 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff59ff-b5af-4d73-a488-22f2e9fdacf0 service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:32:35.134707 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff59ff-b5af-4d73-a488-22f2e9fdacf0 service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:32:35.134981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff59ff-b5af-4d73-a488-22f2e9fdacf0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:32:35.137120 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-7dff59ff-b5af-4d73-a488-22f2e9fdacf0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:32:35.150979 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff59ff-b5af-4d73-a488-22f2e9fdacf0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:32:35.152656 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff59ff-b5af-4d73-a488-22f2e9fdacf0 service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:32:35.163183 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1460/2925] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:32:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:32:43.928731 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef961c-c338-4ff0-a25d-60811bda177f None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:32:43.929243 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef961c-c338-4ff0-a25d-60811bda177f None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:32:43.929351 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef961c-c338-4ff0-a25d-60811bda177f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:32:43.950944 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-7bef961c-c338-4ff0-a25d-60811bda177f None None] MFA Rules not processed for user `884ed6ddb71647088b33dc9623143b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:32:43.952208 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7bef961c-c338-4ff0-a25d-60811bda177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:32:43.968854 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1466/2926] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 4 06:32:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:32:44.008439 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff59ff-b5af-4d73-a488-22f2e9fdacf0 service placement] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:32:44.020140 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff59ff-b5af-4d73-a488-22f2e9fdacf0 service placement] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:32:44.025440 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49b037-e9a8-43c2-a40d-026b6132d7d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655b3c5b98fa4d91b328d04d42fc263a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:32:44.026084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49b037-e9a8-43c2-a40d-026b6132d7d6 service swift] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:32:44.026364 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49b037-e9a8-43c2-a40d-026b6132d7d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:32:44.026625 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49b037-e9a8-43c2-a40d-026b6132d7d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:32:44.034616 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49b037-e9a8-43c2-a40d-026b6132d7d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:32:44.036236 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49b037-e9a8-43c2-a40d-026b6132d7d6 service swift] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:32:44.044309 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1461/2927] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 4 06:32:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 06:32:50.525404 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef961c-c338-4ff0-a25d-60811bda177f None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:32:50.527227 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-7bef961c-c338-4ff0-a25d-60811bda177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:32:50.595847 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef961c-c338-4ff0-a25d-60811bda177f None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:32:50.600464 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:32:50.601348 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:32:50.601625 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:32:50.601897 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] PATH_INFO: `/v3/users/82e1f29c9cda427cac832f4481fd0b26` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:32:50.603149 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e1f29c9cda427cac832f4481fd0b26)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:32:50.605119 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:32:50.655294 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e1f29c9cda427cac832f4481fd0b26'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:50.661978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e1f29c9cda427cac832f4481fd0b26'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:50.662699 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e1f29c9cda427cac832f4481fd0b26'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:50.671962 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e1f29c9cda427cac832f4481fd0b26'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:50.684264 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e1f29c9cda427cac832f4481fd0b26'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:50.690660 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1467/2928] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:32:50 2022] DELETE /identity/v3/users/82e1f29c9cda427cac832f4481fd0b26 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:32:50.869402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49b037-e9a8-43c2-a40d-026b6132d7d6 service swift] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:32:50.881197 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49b037-e9a8-43c2-a40d-026b6132d7d6 service swift] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:32:50.938423 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:32:50.939337 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:32:50.939603 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:32:50.939876 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] PATH_INFO: `/v3/projects/764c6dcba64244dba2aff169c181ccea` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:32:50.941067 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=764c6dcba64244dba2aff169c181ccea)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:32:50.942970 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:32:50.992700 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764c6dcba64244dba2aff169c181ccea', 'request_id': 'req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:50.999709 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764c6dcba64244dba2aff169c181ccea', 'request_id': 'req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.005284 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764c6dcba64244dba2aff169c181ccea', 'request_id': 'req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.013349 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:32:51.020985 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] The token cache is being invalidate because project 764c6dcba64244dba2aff169c181ccea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:32:51.021473 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.022544 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.024490 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1462/2929] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:32:50 2022] DELETE /identity/v3/projects/764c6dcba64244dba2aff169c181ccea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:32:51.028555 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:32:51.029998 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:32:51.043283 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266c2df-1a07-44bb-b861-a94efa2d8d3a admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:32:51.103185 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:32:51.104139 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:32:51.104430 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:32:51.104701 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] PATH_INFO: `/v3/users/8f25016c9ef74db18317f03bf018d409` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:32:51.105920 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f25016c9ef74db18317f03bf018d409)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:32:51.107934 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:32:51.113260 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:32:51.156225 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f25016c9ef74db18317f03bf018d409'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.163201 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f25016c9ef74db18317f03bf018d409'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.164300 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f25016c9ef74db18317f03bf018d409'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.173574 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f25016c9ef74db18317f03bf018d409'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.186596 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-13461a5d-c9ec-44cd-8dcb-93f21dc4f237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f25016c9ef74db18317f03bf018d409'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.192669 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1468/2930] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:32:51 2022] DELETE /identity/v3/users/8f25016c9ef74db18317f03bf018d409 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:32:51.376831 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:32:51.378275 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:32:51.392894 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d1d21-2575-4e4d-b9dc-cae74e6521c2 admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:32:51.453074 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:32:51.454086 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:32:51.454394 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:32:51.454667 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] PATH_INFO: `/v3/projects/ca6c7523fe114d4ab10abade04851059` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:32:51.455959 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6c7523fe114d4ab10abade04851059)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:32:51.457930 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:32:51.511759 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6c7523fe114d4ab10abade04851059', 'request_id': 'req-b46c7dbe-4369-4472-a313-c65c59780de5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.518080 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6c7523fe114d4ab10abade04851059', 'request_id': 'req-b46c7dbe-4369-4472-a313-c65c59780de5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.523969 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6c7523fe114d4ab10abade04851059', 'request_id': 'req-b46c7dbe-4369-4472-a313-c65c59780de5'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.540070 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] The token cache is being invalidate because project ca6c7523fe114d4ab10abade04851059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:32:51.540354 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.541043 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-b46c7dbe-4369-4472-a313-c65c59780de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:32:51.542825 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1463/2931] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:32:51 2022] DELETE /identity/v3/projects/ca6c7523fe114d4ab10abade04851059 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:33:01.189031 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:33:01.537228 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:33:35.124524 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086c11d-332b-4c7c-acdc-bf0805662953 None None] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:33:35.125168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086c11d-332b-4c7c-acdc-bf0805662953 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:33:35.125168 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086c11d-332b-4c7c-acdc-bf0805662953 None None] PATH_INFO: `/` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:33:35.126477 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1469/2932] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 06:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 06:33:35.132378 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f4aa-0a10-45ce-81e4-2a08de6cd3b3 None None] REQUEST_METHOD: `POST` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:33:35.132869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f4aa-0a10-45ce-81e4-2a08de6cd3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:33:35.132869 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f4aa-0a10-45ce-81e4-2a08de6cd3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:33:35.182780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.auth.core [None req-ecd4f4aa-0a10-45ce-81e4-2a08de6cd3b3 None None] MFA Rules not processed for user `817b1a026f4b443daea514338ad8531a`. Rule list: `[]` (Enabled: `True`). {{(pid=80129) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:33:35.184077 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-ecd4f4aa-0a10-45ce-81e4-2a08de6cd3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:33:35.240758 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1464/2933] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 06:33:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:33:35.254984 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086c11d-332b-4c7c-acdc-bf0805662953 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:33:35.256744 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-f086c11d-332b-4c7c-acdc-bf0805662953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:33:35.270681 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086c11d-332b-4c7c-acdc-bf0805662953 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:33:35.274329 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28191e18-65a8-4855-bf0b-8ce55cd9b49c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:33:35.274966 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28191e18-65a8-4855-bf0b-8ce55cd9b49c service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:33:35.275273 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28191e18-65a8-4855-bf0b-8ce55cd9b49c service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:33:35.275644 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28191e18-65a8-4855-bf0b-8ce55cd9b49c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:33:35.282803 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28191e18-65a8-4855-bf0b-8ce55cd9b49c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:33:35.284262 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28191e18-65a8-4855-bf0b-8ce55cd9b49c service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:33:35.291961 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1470/2934] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:33:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:33:35.381324 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befae392-ee8e-4e70-a816-538f9d5b5f63 None None] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:33:35.381621 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befae392-ee8e-4e70-a816-538f9d5b5f63 None None] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:33:35.381893 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befae392-ee8e-4e70-a816-538f9d5b5f63 None None] PATH_INFO: `/` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:33:35.383388 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1465/2935] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 4 06:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 06:33:35.389013 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade857a-d6fa-42c9-abf5-7db9621e62a9 None None] REQUEST_METHOD: `POST` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:33:35.389298 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade857a-d6fa-42c9-abf5-7db9621e62a9 None None] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:33:35.389571 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade857a-d6fa-42c9-abf5-7db9621e62a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:33:35.441548 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.auth.core [None req-0ade857a-d6fa-42c9-abf5-7db9621e62a9 None None] MFA Rules not processed for user `2cb245e4300e4d5fbdd425e936f4ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80128) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 06:33:35.442787 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0ade857a-d6fa-42c9-abf5-7db9621e62a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:33:35.530229 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1471/2936] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 4 06:33:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 06:33:35.541107 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befae392-ee8e-4e70-a816-538f9d5b5f63 None None] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:33:35.553796 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befae392-ee8e-4e70-a816-538f9d5b5f63 None None] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:33:35.558230 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33782-1c17-42d0-a3ed-f79011ceb4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:33:35.558874 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33782-1c17-42d0-a3ed-f79011ceb4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:33:35.559148 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33782-1c17-42d0-a3ed-f79011ceb4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:33:35.559453 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33782-1c17-42d0-a3ed-f79011ceb4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:33:35.571026 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b33782-1c17-42d0-a3ed-f79011ceb4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:33:35.572501 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b33782-1c17-42d0-a3ed-f79011ceb4f5 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:33:35.583838 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1466/2937] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:33:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:33:45.525391 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=52 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:33:45.578928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=39 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:35:02.620082 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade857a-d6fa-42c9-abf5-7db9621e62a9 None None] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:35:02.621745 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-0ade857a-d6fa-42c9-abf5-7db9621e62a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:35:02.637084 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade857a-d6fa-42c9-abf5-7db9621e62a9 None None] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:35:02.645117 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1c995-6868-4c2a-9285-7a2b2ecdbb39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb245e4300e4d5fbdd425e936f4ac77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:35:02.645322 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1c995-6868-4c2a-9285-7a2b2ecdbb39 service nova] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:35:02.645605 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1c995-6868-4c2a-9285-7a2b2ecdbb39 service nova] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:35:02.645883 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1c995-6868-4c2a-9285-7a2b2ecdbb39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:35:02.647732 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-33c1c995-6868-4c2a-9285-7a2b2ecdbb39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:35:02.661424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1c995-6868-4c2a-9285-7a2b2ecdbb39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:35:02.664601 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1c995-6868-4c2a-9285-7a2b2ecdbb39 service nova] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:35:02.676466 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1472/2938] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:35:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:35:02.812981 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33782-1c17-42d0-a3ed-f79011ceb4f5 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:35:02.822191 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33782-1c17-42d0-a3ed-f79011ceb4f5 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:35:02.826402 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdaad0-6db3-4eaa-ad08-aa2e65a2b279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:35:02.827084 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdaad0-6db3-4eaa-ad08-aa2e65a2b279 service neutron] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:35:02.827411 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdaad0-6db3-4eaa-ad08-aa2e65a2b279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:35:02.827691 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdaad0-6db3-4eaa-ad08-aa2e65a2b279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:35:02.829499 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-9ecdaad0-6db3-4eaa-ad08-aa2e65a2b279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:35:02.843706 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdaad0-6db3-4eaa-ad08-aa2e65a2b279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:35:02.845160 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdaad0-6db3-4eaa-ad08-aa2e65a2b279 service neutron] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:35:02.856494 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1467/2939] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:35:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:35:04.038666 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1c995-6868-4c2a-9285-7a2b2ecdbb39 service nova] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:35:04.040349 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-33c1c995-6868-4c2a-9285-7a2b2ecdbb39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:35:04.052310 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1c995-6868-4c2a-9285-7a2b2ecdbb39 service nova] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:35:04.056295 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c751694-cd25-4bb3-8401-511761aeb8d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817b1a026f4b443daea514338ad8531a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:35:04.056891 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c751694-cd25-4bb3-8401-511761aeb8d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:35:04.057204 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c751694-cd25-4bb3-8401-511761aeb8d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:35:04.057471 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c751694-cd25-4bb3-8401-511761aeb8d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:35:04.059280 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4c751694-cd25-4bb3-8401-511761aeb8d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:35:04.072077 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c751694-cd25-4bb3-8401-511761aeb8d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:35:04.073511 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c751694-cd25-4bb3-8401-511761aeb8d2 service neutron] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:35:04.084651 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1473/2940] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:35:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:35:09.760222 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdaad0-6db3-4eaa-ad08-aa2e65a2b279 service neutron] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:35:09.762048 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-9ecdaad0-6db3-4eaa-ad08-aa2e65a2b279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:35:09.839790 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdaad0-6db3-4eaa-ad08-aa2e65a2b279 service neutron] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:35:09.844936 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea791c-9eee-45d2-8c4a-8bc984c92a2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:35:09.845602 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea791c-9eee-45d2-8c4a-8bc984c92a2a service placement] REQUEST_METHOD: `GET` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:35:09.845886 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea791c-9eee-45d2-8c4a-8bc984c92a2a service placement] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:35:09.846160 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea791c-9eee-45d2-8c4a-8bc984c92a2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:35:09.848001 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-a8ea791c-9eee-45d2-8c4a-8bc984c92a2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:35:09.860081 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea791c-9eee-45d2-8c4a-8bc984c92a2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:35:09.861693 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea791c-9eee-45d2-8c4a-8bc984c92a2a service placement] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:35:09.873708 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1468/2941] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:35:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:35:12.813010 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c751694-cd25-4bb3-8401-511761aeb8d2 service neutron] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:35:12.814519 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4c751694-cd25-4bb3-8401-511761aeb8d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:35:12.882606 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c751694-cd25-4bb3-8401-511761aeb8d2 service neutron] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:35:12.887932 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:35:12.888799 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:35:12.889074 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:35:12.889345 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] PATH_INFO: `/v3/users/dcfeaad253164dfe96a0ff1c3a578609` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:35:12.890657 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcfeaad253164dfe96a0ff1c3a578609)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:35:12.892632 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:35:12.946025 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcfeaad253164dfe96a0ff1c3a578609'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:12.952804 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcfeaad253164dfe96a0ff1c3a578609'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:12.953452 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcfeaad253164dfe96a0ff1c3a578609'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:12.962693 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcfeaad253164dfe96a0ff1c3a578609'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:12.974687 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcfeaad253164dfe96a0ff1c3a578609'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:12.980792 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1474/2942] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:35:12 2022] DELETE /identity/v3/users/dcfeaad253164dfe96a0ff1c3a578609 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:35:13.161097 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea791c-9eee-45d2-8c4a-8bc984c92a2a service placement] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:35:13.181903 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea791c-9eee-45d2-8c4a-8bc984c92a2a service placement] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:35:13.241713 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:35:13.242951 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:35:13.243235 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:35:13.243548 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] PATH_INFO: `/v3/projects/e72c59146021441f836a3b822b9c1d1e` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:35:13.244764 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e72c59146021441f836a3b822b9c1d1e)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:35:13.246927 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:35:13.296928 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e72c59146021441f836a3b822b9c1d1e', 'request_id': 'req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.302913 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e72c59146021441f836a3b822b9c1d1e', 'request_id': 'req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.308334 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e72c59146021441f836a3b822b9c1d1e', 'request_id': 'req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.323678 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] The token cache is being invalidate because project e72c59146021441f836a3b822b9c1d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:35:13.323962 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.324572 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.326147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1469/2943] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:35:13 2022] DELETE /identity/v3/projects/e72c59146021441f836a3b822b9c1d1e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:35:13.330374 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:35:13.331858 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:35:13.345654 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa67208-d831-429b-ab11-e7ff0d98d832 admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:35:13.396018 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:35:13.406172 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:35:13.407063 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:35:13.407562 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:35:13.408061 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] PATH_INFO: `/v3/users/c936b59e3c6f48478c30de3d3dd7a34b` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:35:13.409542 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c936b59e3c6f48478c30de3d3dd7a34b)` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:35:13.411510 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:35:13.460621 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c936b59e3c6f48478c30de3d3dd7a34b'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.467978 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c936b59e3c6f48478c30de3d3dd7a34b'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.468800 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c936b59e3c6f48478c30de3d3dd7a34b'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.477169 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c936b59e3c6f48478c30de3d3dd7a34b'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.489214 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.notifications [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c936b59e3c6f48478c30de3d3dd7a34b'} {{(pid=80128) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.495424 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1475/2944] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jul 4 06:35:13 2022] DELETE /identity/v3/users/c936b59e3c6f48478c30de3d3dd7a34b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:35:13.684441 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] Authenticating user token {{(pid=80129) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:35:13.686134 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.fernet_utils [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80129) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:35:13.701066 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a0fb2-3b82-48f4-8edd-80bbc7ccd03b admin admin] Validating token access rules against request {{(pid=80129) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:35:13.705586 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:35:13.774094 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825e62d400e54cc0868912432c257354', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4d1bfa73f141cd90a91a1d9547a701', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80129) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:35:13.775636 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:35:13.775964 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:35:13.776358 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] PATH_INFO: `/v3/projects/7cec5546b49e4dde8dbe6c44947790e2` {{(pid=80129) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:35:13.778263 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cec5546b49e4dde8dbe6c44947790e2)` {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:35:13.782147 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] RBAC: Authorization granted {{(pid=80129) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:35:13.839436 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cec5546b49e4dde8dbe6c44947790e2', 'request_id': 'req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.846146 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cec5546b49e4dde8dbe6c44947790e2', 'request_id': 'req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.852194 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cec5546b49e4dde8dbe6c44947790e2', 'request_id': 'req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf'} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.866813 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] The token cache is being invalidate because project 7cec5546b49e4dde8dbe6c44947790e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80129) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 06:35:13.867105 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.868150 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG keystone.notifications [None req-1373c06b-34d7-40e8-8cfb-4d706c1d72bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80129) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 06:35:13.869780 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: [pid: 80129|app: 0|req: 1470/2945] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jul 4 06:35:13 2022] DELETE /identity/v3/projects/7cec5546b49e4dde8dbe6c44947790e2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 06:35:23.491974 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:35:23.863820 dv-cinder-797333-5-iscsi devstack@keystone.service[80129]: DEBUG dbcounter [-] [80129] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80129) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 06:37:38.007928 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] Authenticating user token {{(pid=80128) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 06:37:38.010207 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:37:38.130776 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aac45e-80a3-4fa4-a396-828c785a49ea admin admin] Validating token access rules against request {{(pid=80128) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 06:37:38.192881 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae96a2-039f-4309-8a54-f9849f2df957 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '486de932d1db422ba2c3d84142f9a056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b95f166a586c4a5fa22e04f9a5dc55fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80128) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 06:37:38.193541 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae96a2-039f-4309-8a54-f9849f2df957 service placement] REQUEST_METHOD: `GET` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 06:37:38.193825 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae96a2-039f-4309-8a54-f9849f2df957 service placement] SCRIPT_NAME: `/identity` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 06:37:38.194073 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae96a2-039f-4309-8a54-f9849f2df957 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80128) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 06:37:38.195908 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.fernet_utils [None req-9fae96a2-039f-4309-8a54-f9849f2df957 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80128) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 06:37:38.207354 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae96a2-039f-4309-8a54-f9849f2df957 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 06:37:38.208947 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae96a2-039f-4309-8a54-f9849f2df957 service placement] RBAC: Authorization granted {{(pid=80128) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 06:37:38.219714 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: [pid: 80128|app: 0|req: 1476/2946] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 4 06:37:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 06:37:48.215007 dv-cinder-797333-5-iscsi devstack@keystone.service[80128]: DEBUG dbcounter [-] [80128] Writing DB stats keystone:SELECT=75 {{(pid=80128) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}